bilibili/Index-Translate

选用指南

AI 根据 README 整理 · 2026/10/17 05:40:53

使用方式

可通过免费在线 API 调用，支持 OpenAI 兼容接口，也可本地部署 vLLM 或使用 llama.cpp 进行推理。

查看 README 依据

> 🚀 **Free Public API Now Available!** Call **Index-Translate-35B-A3B** directly with zero GPU setup. Fully OpenAI-compatible at `https://index-translate.bilibili.com/v1`. Try it in seconds with `python inference/llm/call_api.py "Hello, world!" --target zh`! 👉 [API Quick Start](#option-1-free-online-api-zero-gpu-setup)

> 👉 **[【Setup Guide】Immersive Translate Configuration Tutorial (Windows / macOS / Linux, with Screenshots & Troubleshooting)](docs/immersive_translate.md)** #### Option 2: Self-hosted local vLLM Start with the 2B text model on a CUDA GPU using a vLLM build with Qwen3.5 support. From a terminal: ```bash git clone https://github.com/bilibili/Index-Translate.git cd Index-Translate pip install -U vllm pip install -r inference/llm/requirements.txt vllm serve IndexTeam/Index-Translate-2B --host 127.0.0.1 --port 8000 --max-model-len 4096 ```

部署要求

本地部署需要 CUDA GPU 和 vLLM 支持 Qwen3.5 的版本，或使用 llama.cpp 进行 GGUF 推理。

查看 README 依据

> 👉 **[【Setup Guide】Immersive Translate Configuration Tutorial (Windows / macOS / Linux, with Screenshots & Troubleshooting)](docs/immersive_translate.md)** #### Option 2: Self-hosted local vLLM Start with the 2B text model on a CUDA GPU using a vLLM build with Qwen3.5 support. From a terminal: ```bash git clone https://github.com/bilibili/Index-Translate.git cd Index-Translate pip install -U vllm pip install -r inference/llm/requirements.txt vllm serve IndexTeam/Index-Translate-2B --host 127.0.0.1 --port 8000 --max-model-len 4096 ```

- **2026-10-03:** released official quantized builds across the family on Hugging Face and ModelScope — **GGUF** for llama.cpp local inference (including visual `mmproj` weights), alongside **FP8** (W8A8) and **NVFP4** (W4A4, Blackwell-optimized) for vLLM serving. - **2026-09-30:** released Index-Translate, with 2B / 9B / 35B-A3B (preview) text-model weights on Hugging Face and ModelScope, the technical report, and the online demo. ## TODO - [x] Release official quantized builds (GGUF for llama.cpp local inference, FP8 and NVFP4 for vLLM serving) for the whole family.

已知限制

Index-Echo S2ST 的 GGUF 仓库仅包含文本 LLM 骨干，完整流水线需使用 FP8/FP4 版本；Index-NativeLong 的模型 ID 为 Index-Nailong。

查看 README 依据

**Quantized builds:** every model above is also published in **GGUF** (llama.cpp local inference; all bit-widths in one repository per model, vision mmproj included where applicable), **FP8** (compressed-tensors W8A8, ready for vLLM serving), and **FP4** (compressed-tensors NVFP4 W4A4, for Blackwell GPUs). For the Index-Echo speech models, the GGUF repositories contain the **text LLM backbone only**, while the FP8/FP4 repositories ship the **complete pipeline** with a quantized LLM. All repositories are mirrored on [ModelScope](https://modelscope.cn/organization/IndexTeam).

**Naming:** Index-NativeLong is published under the model IDs `IndexTeam/Index-Nailong-2B` and `IndexTeam/Index-Nailong-9B`. Use those IDs in commands. Language support for the speech and long-document packages is listed separately from the text models' 150-language coverage.