nokia-applied-research/AnyJev

选用指南

AI 根据 README 整理 · 2026/10/16 20:12:00

使用方式

通过 Python 库使用，支持 transformers、vLLM 和 HTTP 网关。

查看 README 依据

```python from anyjev import Tacit tacit = Tacit.from_pretrained("morriszjm/Tacit-9B") # transformers, one GPU d = tacit.decide(state="Customer: my package was due last Monday and it still has not arrived.", question="What does the customer want?", options=["track_order", "cancel_order", "refund", "change_address"]) d["answer"], d["probs"], d["route"] # an option, {option: probability}, "one_forward" or "cot" ``` `kind` is `"choice"` (default), `"yes_no"` or `"score"` (`options` are ordered levels, lowest

escalate, however hard the traffic gets. Every escalation is reported (`route == "cot"`, with the first pass kept), and `tacit.stats` counts them. **On vLLM.** A stock server; the client needs only the tokenizer. ```bash pip install "anyjev[client]" vllm serve morriszjm/Tacit-9B --host 127.0.0.1 --port 8000 ``` ```python tacit = Tacit.from_pretrained("morriszjm/Tacit-9B", engine="server", base_url="http://127.0.0.1:8000", adaptive=True) ``` `engine="vllm"` (`pip install "anyjev[vllm]"`) runs vLLM in the same process instead.

**An HTTP endpoint for many clients sharing one cap.** ```bash python -m anyjev.serve --model morriszjm/Tacit-9B --upstream http://127.0.0.1:8000 --adaptive --port 8100 curl -s 127.0.0.1:8100/v1/decide -H 'Content-Type: application/json' \ -d '{"state": "Customer: my package has not arrived.", "question": "What does the customer want?", "options": ["track_order", "refund"]}' ``` `POST /v1/decide` takes one decision or `{"items": [...]}`; `GET /v1/stats` reports the escalation share. The gateway listens on 127.0.0.1 and has no authentication of its own. | argument | default | meaning |

部署要求

需要 Python 环境，Tacit 模型需 GPU，vLLM 服务需相应硬件。

查看 README 依据

```python from anyjev import Tacit tacit = Tacit.from_pretrained("morriszjm/Tacit-9B") # transformers, one GPU d = tacit.decide(state="Customer: my package was due last Monday and it still has not arrived.", question="What does the customer want?", options=["track_order", "cancel_order", "refund", "change_address"]) d["answer"], d["probs"], d["route"] # an option, {option: probability}, "one_forward" or "cot" ``` `kind` is `"choice"` (default), `"yes_no"` or `"score"` (`options` are ordered levels, lowest

escalate, however hard the traffic gets. Every escalation is reported (`route == "cot"`, with the first pass kept), and `tacit.stats` counts them. **On vLLM.** A stock server; the client needs only the tokenizer. ```bash pip install "anyjev[client]" vllm serve morriszjm/Tacit-9B --host 127.0.0.1 --port 8000 ``` ```python tacit = Tacit.from_pretrained("morriszjm/Tacit-9B", engine="server", base_url="http://127.0.0.1:8000", adaptive=True) ``` `engine="vllm"` (`pip install "anyjev[vllm]"`) runs vLLM in the same process instead.

已知限制

作者指出存在已知限制，详见 docs/limitations.md。

查看 README 依据

Dated plan and help-wanted files: [ROADMAP.md](https://github.com/nokia-applied-research/AnyJev/blob/main/ROADMAP.md). Known limitations: [docs/limitations.md](https://github.com/nokia-applied-research/AnyJev/blob/main/docs/limitations.md). ## 🤝 Contributing and citation