Betelgeuse-AI/Betelgeuse

选用指南

AI 根据 README 整理 · 2026/11/1 12:32:11

使用方式

通过编译后的 `bel-server` 或 `bel-cli` 可执行文件使用，支持 OpenAI 兼容 API 和交互式命令行聊天。

查看 README 依据

Betelgeuse keeps dense computation on the GPU and brings routed experts into a bounded VRAM pool only when needed. This is native C++/CUDA work, not a launcher around an upstream binary. The executables are **`bel-server`** and **`bel-cli`**; the llama C API and internal build target names remain unchanged.

Open `http://127.0.0.1:8080` or use the OpenAI-compatible API. The systemd scope enforces the historical 12 GiB process and zero-swap limits on the 16 GB host; it does not allocate extra memory. Keep the server bound to localhost unless you have separately reviewed authentication and deployment security. For direct interactive CLI chat, use the same memory envelope and paging parameters. No separately started server or `--server-base` is needed: ```bash systemd-run --user --scope --quiet -p MemoryMax=12G -p MemorySwapMax=0 \ build/bin/bel-cli \

部署要求

需要 Linux x86-64 系统、支持 CUDA sm_86 的 NVIDIA GPU（如 RTX 3060 12GB）、16GB 系统内存、本地 NVMe 存储，以及约 68GB 的磁盘空间用于模型文件。

查看 README 依据

not a 3.2 GB dense-only pruning scheme. Original two-shard GGUFs and pack-building indexes are conversion inputs, not extra runtime requirements. ## Build native binaries The paged configuration targets **Linux x86-64, CUDA `sm_86`, RTX 3060 12 GB and local NVMe**. Install a C++17 compiler, CMake, CUDA toolkit/driver, liburing development files and the normal [llama.cpp build dependencies](docs/build.md). Building the embedded terminal logo also requires Python 3, `rsvg-convert` (librsvg) and ImageMagick (`magick` or `convert`); the installed binaries need none of these image tools at

--local-dir /path/to/Alpha-Orionis-XS export BUNDLE=/path/to/Alpha-Orionis-XS ``` The profile seeds likely hot experts; it is neither weights nor a prewarmed cache snapshot. For the tested source files, the dense GGUF is about **3.77 GB** (3.50 GiB of dense tensor payload plus metadata), the per-expert-aligned `experts.bin` layout requires **35,481,714,688 bytes** (35.48 GB), the unchanged PLE companion is **28,800,138,432 bytes** (26.82 GiB), and the profile is **6,920 bytes**. Allow roughly 68 GB of storage for the complete runtime bundle. All non-expert tensors are retained; this is

已知限制

作者明确指出该项目是实验性的，部分功能可能损坏；仅支持单模型上下文，且自定义 CUDA 路径仅针对 sm_86 架构，其他 GPU 或 CPU 构建未经验证。

查看 README 依据

> **Experimental and partly broken. Prefer `bel-server`.** The paged path is hardware-specific, not a production release or a general promise that any large MoE will fit or run well. CLI behavior, unfamiliar models and configurations outside the documented envelope need independent validation. No MTP configuration is certified for exact-output paging. ## What the numbers mean | Result | Token generation | Evidence and limits | | --- | ---: | --- | | Upstream llama.cpp baseline | **0.9 tok/s** | User-reported on the same hardware; not a new controlled benchmark here. |

- One paged model context per process. Keep `-np 1`; general server multi-user features do not make shared paging state safe for parallel contexts. - The custom CUDA path targets `sm_86`. Other GPUs, CPU-only builds, non-Linux systems and other backends are not certified for paging. - The recorded envelope is `-c 512 -ub 64 -t 6 -ngl 99`, with effective eight-token paging chunks. Larger contexts and different batch sizes need their own memory/correctness checks.