incoai/splash

选用指南

AI 根据 README 整理 · 2026/10/16 21:17:00

使用方式

通过命令行启动服务，使用 `splash serve` 指定模型，然后可通过浏览器访问内置聊天页面或使用 `splash opencode` 等命令运行编码代理。也提供 OpenAI 和 Anthropic 兼容的 API。

查看 README 依据

decoding, specialized Metal kernels, and automatic memory planning, with vision, tool calling, and a built-in chat page. It reuses cached prefixes and batches concurrent requests automatically. ## Quick start Apple M3 or newer, macOS 26.4 or later, and [Homebrew](https://brew.sh). The 4-bit examples need at least 36 GB of unified memory (48 GB recommended); 24 GB Macs can use [smaller GGUF variants](#models). ```bash brew install incoai/tap/splash splash serve --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M ``` The first run downloads the model and its matching draft and starts serving

on `127.0.0.1:8000`. Later starts reuse the downloads; leave room on disk for them ([storage requirements](DEVELOPMENT.md#model-storage)). Once it prints `Ready`, leave this terminal open. Open <http://127.0.0.1:8000> in your browser, or run an installed coding agent from another terminal: ```bash splash opencode # or: splash claude / splash codex / splash hermes / splash pi ``` For a server started with another `--port`, pass the same `--port` to the agent. Press Ctrl+C in the server terminal to stop Splash.

For LM Studio Bionic, follow its [Splash setup guide](https://lmstudio.ai/blog/splash-engine). ## Use the API OpenAI Chat Completions, Responses and Completions, and Anthropic Messages, with streaming, tool calls, JSON Schema output, and base64 images and PDFs: ```bash curl http://127.0.0.1:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model": "unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M", "messages": [{"role": "user", "content": "Explain speculative decoding in one sentence."}] }' ```

部署要求

需要 Apple M3 或更新芯片、macOS 26.4 或更高版本，以及 Homebrew。4-bit 示例至少需要 36 GB 统一内存（推荐 48 GB），24 GB Mac 可使用较小的 GGUF 变体。

查看 README 依据

decoding, specialized Metal kernels, and automatic memory planning, with vision, tool calling, and a built-in chat page. It reuses cached prefixes and batches concurrent requests automatically. ## Quick start Apple M3 or newer, macOS 26.4 or later, and [Homebrew](https://brew.sh). The 4-bit examples need at least 36 GB of unified memory (48 GB recommended); 24 GB Macs can use [smaller GGUF variants](#models). ```bash brew install incoai/tap/splash splash serve --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M ``` The first run downloads the model and its matching draft and starts serving

已知限制

不支持全 BF16 和 F16 模型，安装时会拒绝此类 GGUF。服务器默认在 localhost 上监听且无认证。

查看 README 依据

| Qwen3.6-35B-A3B | `unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M` | `mlx-community/Qwen3.6-35B-A3B-4bit` | Unsloth GGUF variants span **1–8 bits**, including mixed-precision UD formats up to `UD-Q8_K_XL`, whose BF16 tensors load too. Whole-BF16 and F16 models are not supported: the installer refuses a GGUF whose projections and experts are all BF16, F16 or F32. [Prism ML Ternary Bonsai 2](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf) is also supported in PQ2_0 (7.2 GB), including vision. Pass `OWNER/REPO:VARIANT` to `--model`, as in the quick start. MLX targets can be affine 2–8-bit,

| `--disable-ane` | Prefill on the GPU alone. Default: the 27B also uses the Neural Engine. | | `--max-cache-disk 16G` | Cap the SSD cache (default: auto, up to 32 GiB; `0` turns it off). | | `--persistent-cache` | Keep the SSD cache across restarts. Off by default. | Use `--max-memory` to leave room for other applications. The server listens on localhost without authentication by default. For LAN access, authentication, browser apps on other origins, and other options, see [server configuration](DEVELOPMENT.md#server-configuration) or `splash serve --help`.