bespokelabsai/nimble

选用指南

AI 根据 README 整理 · 2026/10/17 02:50:23

使用方式

通过 Python 代码使用，加载模型后调用 scorer.score() 进行决策。

查看 README 依据

processes the shared context again for each field, while the MLX runner processes it only once. ### Make a typed decision After you load either scorer, continue in the same Python session. ```python schema = { "priority": { "type": "enum", "choices": ["HIGH", "LOW"], "description": "Urgency based on current business impact.", "choice_descriptions": { "HIGH": "A critical business operation is currently blocked.", "LOW": "An optional enhancement with no current business impact.", }, }, "requires_review": {

"type": "boolean", "description": "Whether customers are unable to complete a purchase.", }, } result = scorer.score("The payment service is down for all customers.", schema) print(result["output"]) print(result["fields"]["priority"]["scores"]) # Candidate probabilities. print(result["fields"]["priority"]["logits"]) ``` The output has this shape. The values depend on the model: ```json {"priority": "HIGH", "requires_review": true} ``` Load the scorer once and reuse it for each new context. The latest checkpoint

部署要求

需要 Python 3.12，在 Mac 上需要 Apple Silicon，在 Linux 上需要支持 BF16 的 NVIDIA GPU。未量化时 9B 权重约需 18 GB 内存。

查看 README 依据

Nimble’s performance depends on the curated data and domains represented in its training; test it on your own tasks. But we do see that Nimble is overall better than its base model Qwen3.5-9B in new domains. ## Quickstart Clone the repository and move into its folder. Run all the commands below from this folder. ```sh git clone https://github.com/bespokelabsai/nimble.git nimble cd nimble ``` Use Python 3.12. To run the model on a Mac, you need Apple Silicon. Python must also run directly on macOS so that it can use Metal, which is Apple's interface

to the GPU. To run the model on Linux, you need an NVIDIA GPU that supports BF16, a 16-bit number format. Without quantization, the 9B weights alone take about 18 GB. Quantization means storing the weights with fewer bits to save memory. The model needs more memory than this while it runs. The merge step below runs on the CPU. It needs extra RAM, and it needs disk space for both the base weights and the merged weights. A Mac with 64 GB of memory has more free memory for this than a machine with 24 GB. ### Download the model

已知限制

仅接受文本输入，不能处理图像；只能从提供的答案中选择，不能生成文本或嵌套 JSON；每个提示最多 8192 个 token；概率不保证正确性。

查看 README 依据

the fields in parallel. The CUDA scorer scores each field on its own, with the full prompt each time. Both scorers return the typed output. They also return the logits and the probabilities of the candidate answers. Each field is scored separately, so one field cannot see the answer to another field. ### What you cannot build with the current release - Nimble accepts only text. You cannot use it to judge other kinds of input, e.g., images. This is true even though the base model includes a vision part. - Nimble only picks from the answers you supply. It cannot write text of its

own, e.g., an explanation. It also cannot return nested JSON or a piece of text taken from the context. An enum field can have 1 to 255 string choices in the latest release, and a boolean field has two. - The probabilities are not a guarantee that an answer is correct. Nimble scales them so that they add up to 1 across the answers you supplied. The latest checkpoint uses T=1.0 and has not had a separate temperature fit. Earlier releases have their own temperature settings. See [Probability temperature](#probability-temperature). A probability

of 0.9 still does not mean that the answer is right 90% of the time on your data. If it is possible that none of your answers fit, add an answer that means "no match". Test any probability threshold on your own data before you rely on it. - Each prompt can have at most 8,192 tokens in the latest release. This limit includes the schema and the part of the prompt that names the field to score. Nimble rejects longer prompts. Fields cannot depend on each other, so your code must check that the answers to different fields are consistent.