MiaAI-Lab/DeepSeek-v4.1-Flash-1017-2x-DGX-Sparks

选用指南

AI 根据 README 整理 · 2026/10/20 00:32:00

使用方式

通过命令行启动，使用OpenAI兼容API访问，支持文本、图像输入和工具调用。

查看 README 依据

leave `HEAD_NFS_SERVER` empty. Set `PREPARE=0` too: `scripts/prepare.sh` assumes a complete cache and would start downloading the 196 GiB checkpoint into a partial one. ## Quick start On the head: ```bash git clone https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-1017-2x-DGX-Sparks.git cd DeepSeek-v4.1-Flash-1017-2x-DGX-Sparks cp scripts/local.sh.example scripts/local.sh # set WORKER=user@192.0.2.2 in it (and FABRIC_PEER, see below) DRY_RUN=1 ./start.sh # prints what it would do; changes and starts nothing ./start.sh ```

step and the server's log, runs a smoke test through both ranks, and prints `DeepSeek-V4.1-Flash-1017 is now LIVE! on port 8888` with the endpoint. Any OpenAI client works with `base_url = "http://<head-address>:8888/v1"` and the model `DeepSeek-V4.1-Flash-1017`. The model thinks before it answers unless a request turns thinking off, so give replies enough `max_tokens`. ```bash ./start.sh restart # restart both ranks, e.g. after changing a setting ./stop.sh # stop both ranks and free their GPU memory

部署要求

需要两台DGX Sparks或GB10系统（各128GB统一内存），直接ConnectX-7链路，Docker及NVIDIA容器运行时，以及足够的磁盘空间（头节点约505GB，工作节点约310GB）。

查看 README 依据

- **Two DGX Sparks** (or two GB10 systems with 128 GB unified memory), with nothing else large on them: the weights take ~98 GiB a Spark and rank 0 holds ~104 GiB on the GPU, so a Spark's memory is the limit, not its disk or its link. `start.sh` refuses to start unless `MIN_AVAIL_GIB` (109) is available on each Spark. A head that also runs a desktop and development tools (~6 GiB here) was left with ~3.1 GiB at the worst point of a 100K-token prompt at the full window and ~3.7 GiB at 524,288 tokens: give the head less to do, or lower `CONTEXT`.

- **The same GPU clock ceiling on both Sparks** (`nvidia-smi -lgc 0,2200`; check with `nvidia-smi -q -d CLOCK`). With TP=2 the ranks meet twice a layer, so the slower GPU sets the pace: one Spark found locked at 1,800 MHz cost about 3-6% of prefill and 3% of decode against 2,200 MHz on both (rigmark, same prompts; the GPUs peaked at 85 / 81 C and nothing throttled). - **A direct ConnectX-7 link:** a QSFP cable between the CX7 ports and an IPv4 address on each end in one private subnet

`ssh-copy-id user@<worker>`; check with `ssh -o BatchMode=yes user@<worker> true`. - Docker with the NVIDIA container runtime and your user in the `docker` group, on both Sparks; `git`, `python3` and the `hf` CLI (`pip install -U huggingface_hub`) on the head. - **Disk.** Head: ~505 GB (the checkpoint 196 GiB, Engram's tables ~203 GB, rank 0's weight file ~106 GB) plus ~25 GB for the image under Docker's root. Worker: ~310 GB (Engram's tables and rank 1's weight file) plus the image; the worker reads the checkpoint from the head over NFS.

已知限制

引擎已知限制：带图像的历史对话每次处理都冷启动，长提示词预填充期间其他流无法解码。

查看 README 依据

Known limits of the engine, from user reports (issues [#2](https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-1017-2x-DGX-Sparks/issues/2) and [#3](https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-1017-2x-DGX-Sparks/issues/3), read in `lanes.zig` and `lane_host.zig` of the pinned engine): a prompt that carries images runs whole and keeps nothing ("a kept state knows no images"), so every turn of a conversation with images in its history is cold (~90 s at 160K tokens in the report); and a prompt pass runs to completion inside its admission, so other streams