Balanov/verdict

选用指南

AI 根据 README 整理 · 2026/11/3 09:06:16

适用场景

适用于需要区分进程退出码与实际执行结果的场景，例如 CI 运行器或工具使用系统，可识别“退出码为0但未运行测试”或“非零退出码但只是环境阻塞”等情况。

查看 README 依据

> **Important:** Verdict is an advisory classifier, not proof that a task or environment mutation completed. Keep deterministic checks and authoritative state verification as the source of truth. ## Why A process can exit successfully while running zero tests, returning an HTTP 500, waiting for a worker, or producing incomplete output. Conversely, a non-zero exit can describe an environment blocker rather than a failed assertion. Agents need that distinction before they retry, stop, or claim completion. ## Benchmark

使用方式

可通过 Python API 调用，也可通过命令行使用，支持指定标准输出、标准错误和权重文件。

查看 README 依据

python -m pip install . ``` ```python from verdict_tiny import Verdict model = Verdict() result = model.predict( command="pytest tests/", exit_code=0, stdout="collected 0 items\nno tests ran in 0.01s", ) print(result) ``` ```json { "label": "unknown", "confidence": 0.999981, "probabilities": { "success": 0.000019, "failure": 0.0, "blocked": 0.0, "unknown": 0.999981 }, "advisory": true } ``` ### CLI ```bash printf 'collected 0 items\nno tests ran\n' | verdict \ --command 'pytest tests/' --exit-code 0 ```

Use `--stdout output.log`, `--stderr error.log`, and `--weights /path/to/verdict-q8.bin` when needed. ### Browser demo The [project page](https://balanov.pro/research/verdict/) runs the same quantized weights in your browser. Logs are not uploaded. ## Output contract | Label | Meaning | |---|---| | `success` | Output contains positive evidence that the operation completed. | | `failure` | The operation ran and reported a concrete failure. | | `blocked` | Permissions, dependencies, network, quota, or environment prevented execution. |

部署要求

推理在本地 CPU 上运行，无 Python 依赖。

查看 README 依据

<div align="center"> <img src="assets/logo.svg" alt="Verdict logo" width="104"> <h1>VERDICT</h1> <p><strong>Exit code 0 is not proof.</strong></p> <p>A 197 KB local model that reads a CLI tool call and returns<br><code>success</code>, <code>failure</code>, <code>blocked</code>, or <code>unknown</code>.</p> </div> Verdict is a research preview of a semantic outcome layer for coding agents, CI runners, and tool-using systems. It looks beyond the process exit code and reads the command, stdout, and stderr. Inference runs locally on CPU with **no Python dependencies**.

已知限制

Verdict 是咨询性分类器，不是任务完成或环境变更的证明，需保留确定性检查作为事实来源。其训练语料为合成数据且偏向英语，基准测试衡量的是对未见模板的泛化能力，而非生产环境代理轨迹的可靠性。

查看 README 依据

> **Important:** Verdict is an advisory classifier, not proof that a task or environment mutation completed. Keep deterministic checks and authoritative state verification as the source of truth. ## Why A process can exit successfully while running zero tests, returning an HTTP 500, waiting for a worker, or producing incomplete output. Conversely, a non-zero exit can describe an environment blocker rather than a failed assertion. Agents need that distinction before they retry, stop, or claim completion. ## Benchmark

Latency is the median of 576 predictions on Windows 11 / AMD64 / Python 3.12, including feature extraction and the Python API call. These are reproducibility measurements, not cross-hardware promises. See [`bench/results.json`](bench/results.json) and reproduce with `python training/train.py`. **The corpus is synthetic and English-biased.** The benchmark measures generalization across held-out templates, not reliability on production agent traces. Real-trace validation is the next milestone. ## Install and run ```bash git clone https://github.com/Balanov/verdict.git cd verdict