StayLameBro/backburner

选用指南

AI 根据 README 整理 · 2026/10/14 19:51:18

适用场景

适用于需要本地运行27B级别大模型，并希望利用iPhone额外算力来加速长上下文（如64k以上）读取和生成的场景。

查看 README 依据

# Backburner Plug your iPhone into your MacBook with a 10 Gb/s USB-C cable and it helps run Qwen3.8-27B locally: - **Faster prefill (up to 64k context).** For every batch of prompt tokens the Mac runs layers 1-40 and the iPhone runs 41-64 on its GPU, pipelined. Your agent waits less every time it reads a file or a tool result of more than ~512 tokens: 29-44% faster prefill at 16k-48k. - **More context.** A 24 GB Mac fits 64k tokens of 8-bit context next to the model. The iPhone holds the oldest part past that

and computes attention over it: its GPU during prefill, its GPU and Neural Engine while writing. The server sizes the total from the phone's free memory at startup (196k-229k tokens at 8-bit on an iPhone 17 Pro Max). Tested end to end to 128k at 8-bit and 140k at 4-bit. - **Same answers.** Greedy output is token-identical with and without the phone (256/256 tokens at 8k and 32k; 32/32 at 140k). The engine is a llama.cpp fork (`llama.cpp/`, [StayLameBro/backburner-llama.cpp](https://github.com/StayLameBro/backburner-llama.cpp))

目标用户

面向拥有Apple Silicon Mac和iPhone 15 Pro或更新机型、iPad Pro，并希望本地运行大模型的开发者或研究人员。

查看 README 依据

cable, through an encrypted, authenticated tunnel ([docs/WIFI.md](docs/WIFI.md)). Nothing is advertised on the network. Before 0.0.3 the app accepted connections over Wi-Fi too: update the app, or keep the phone's Wi-Fi off while Backburner is open. Report problems privately (Security → Report a vulnerability): [SECURITY.md](SECURITY.md). ## Setup You need an Apple Silicon Mac (tested: M4 Pro, 24 GB), an iPhone 15 Pro or newer (tested: 17 Pro Max, 16 Pro Max) or an M-series iPad (untested), and a 10 Gb/s USB-C cable (the cable in the iPhone box is USB 2 and too slow). The app installs

使用方式

通过命令行安装和运行，提供OpenAI兼容的API服务，并需要安装配套的iPhone应用。

查看 README 依据

with a free Apple ID through AltStore, no developer account needed ([docs/INSTALL-IPHONE.md](docs/INSTALL-IPHONE.md)), or builds with Xcode. **One command** downloads the Mac engine and the models (~24 GB) and makes the phone's half. Safe to re-run: ```bash curl -fsSL https://raw.githubusercontent.com/StayLameBro/backburner/main/install.sh | bash backburner phone # once per phone: plug it in, open Backburner, copies the phone's half over the cable backburner # OpenAI-compatible server at http://127.0.0.1:8080/v1 ``` Or step by step: ```bash

部署要求

需要Apple Silicon Mac（测试于M4 Pro 24GB）、iPhone 15 Pro或更新机型（或M系列iPad）、10 Gb/s USB-C线缆。安装需要约24GB模型下载空间，并可能需要调整macOS GPU内存限制。

查看 README 依据

cable, through an encrypted, authenticated tunnel ([docs/WIFI.md](docs/WIFI.md)). Nothing is advertised on the network. Before 0.0.3 the app accepted connections over Wi-Fi too: update the app, or keep the phone's Wi-Fi off while Backburner is open. Report problems privately (Security → Report a vulnerability): [SECURITY.md](SECURITY.md). ## Setup You need an Apple Silicon Mac (tested: M4 Pro, 24 GB), an iPhone 15 Pro or newer (tested: 17 Pro Max, 16 Pro Max) or an M-series iPad (untested), and a 10 Gb/s USB-C cable (the cable in the iPhone box is USB 2 and too slow). The app installs

with a free Apple ID through AltStore, no developer account needed ([docs/INSTALL-IPHONE.md](docs/INSTALL-IPHONE.md)), or builds with Xcode. **One command** downloads the Mac engine and the models (~24 GB) and makes the phone's half. Safe to re-run: ```bash curl -fsSL https://raw.githubusercontent.com/StayLameBro/backburner/main/install.sh | bash backburner phone # once per phone: plug it in, open Backburner, copies the phone's half over the cable backburner # OpenAI-compatible server at http://127.0.0.1:8080/v1 ``` Or step by step: ```bash

# 5. after every reboot: let the GPU keep the model wired (macOS resets this limit) sudo sysctl iogpu.wired_limit_mb=20480 # 6. run: OpenAI-compatible on :8080, uses the iPhone when it's plugged in with Backburner open scripts/serve.sh PHONE=0 scripts/serve.sh # the Mac alone ``` The first `serve.sh` with the phone pushes the Neural Engine page model to it and relaunches the app (about a minute). After that the startup line should read `split prefill on`, `remote KV on` and `ANE pages on`. `scripts/serve.sh` documents each setting next to the measurement that chose it.

已知限制

项目为预发布状态。小读取（小于512 tokens）不会使用手机；超过64k上下文时，手机只负责旧键的注意力计算，不再运行模型层；解码速度在64k以下由Mac决定。手机应用需保持在前台运行，否则服务会停止。

查看 README 依据

footprint after a read: 19.7 -> 18.7 GB (2026-10-01). - **Neural Engine results**, including the two uses we measured and set aside (the Mac's ANE, and the iPhone's ANE for prefill), are in `docs/ANE.md`. ## Limits - **Small reads stay on the Mac.** The phone joins a read of more than ~512 tokens (three 256-token ubatches; the last one always runs on the Mac). Most agent steps are smaller: in a real omp session 7 of 36 requests were big enough, and they carried ~83% of the tokens read.

- **Past 64k the phone does one job, not two.** Its half of the model (layers 41-64) can't see the old keys it holds yet, so past the Mac's 64k cells the Mac runs all 64 layers and the phone only computes the old-key attention. Doing both is the next step. It helps most at 64k-100k; deeper, the phone's GPU is already busy about two thirds of each step with old keys (140k), so a second phone is the bigger win there. - **Writing speed is the Mac's below 64k.** The phone only joins decoding past 64k (attention over the old keys).

- **During a read, a phone failure turns the phone off for 60 s**; the batch reruns on the Mac and the server log says so. - **Past 64k, keep Backburner open and in front on the phone.** The phone then holds the oldest part of the conversation, and the Mac no longer has it. If the phone stops answering for 15 s (the app sent to the background, the screen locked, the cable pulled), the server stops with a message saying so; reopen the app and restart the server. The server log warns after 5 s.