prabeakarve/microgpt

选用指南

AI 根据 README 整理 · 2026/11/5 21:02:01

使用方式

提供CLI，支持train、sample、eval、info、next子命令，也可作为库导入使用。

查看 README 依据

- **Sampling** with temperature, top-k, and a next-token distribution inspector. - **Checkpointing** to compressed `.npz` plus a JSON sidecar with the full config and metrics. - **14 tests**, including a gradient check with a `2e-4` relative-error bound. - **CLI** with `train`, `sample`, `eval`, `info`, and `next` subcommands. ## Install ```bash git clone https://github.com/prabeakarve/microgpt.git cd microgpt python -m pip install -e . ``` Python 3.10+ and NumPy are the only requirements. ## Quick start Train on the bundled corpus (about 8 KB of text, ~60 seconds on a laptop): ```bash

python -m microgpt eval --text examples/corpus.txt ``` ## Library use ```python import numpy as np from microgpt import GPT, ByteTokenizer, TrainConfig, train, complete, perplexity cfg = TrainConfig(corpus="examples/corpus.txt", steps=400, n_embd=96, n_layer=2) meta = train(cfg) # writes cfg.out and cfg.out + ".json" model = GPT(vocab_size=256, block_size=64, n_layer=2, n_head=4, n_embd=96) loss, grads = model.loss_and_grads(idx, targets) # idx, targets: int64 (B, T) print(complete(model, "The model learns", max_new_tokens=120))

部署要求

需要Python 3.10+和NumPy。

查看 README 依据

- **Sampling** with temperature, top-k, and a next-token distribution inspector. - **Checkpointing** to compressed `.npz` plus a JSON sidecar with the full config and metrics. - **14 tests**, including a gradient check with a `2e-4` relative-error bound. - **CLI** with `train`, `sample`, `eval`, `info`, and `next` subcommands. ## Install ```bash git clone https://github.com/prabeakarve/microgpt.git cd microgpt python -m pip install -e . ``` Python 3.10+ and NumPy are the only requirements. ## Quick start Train on the bundled corpus (about 8 KB of text, ~60 seconds on a laptop): ```bash

已知限制

模型是字节级，词汇表固定为256；训练语料较小（约8KB），不适合大规模任务。

查看 README 依据

**Why the mask is applied before the softmax.** Setting masked scores to `-inf` makes their post-softmax probability exactly zero, so no future token can influence the present. Applying a mask afterwards would break the normalization. **Why byte-level tokens.** A learned subword vocabulary is a second thing to train, version, and get wrong. Bytes cost sequence length but remove an entire class of bugs, and they let the model round-trip any UTF-8 text exactly �including text it has never seen. ## Project layout ``` microgpt/

- **Sampling** with temperature, top-k, and a next-token distribution inspector. - **Checkpointing** to compressed `.npz` plus a JSON sidecar with the full config and metrics. - **14 tests**, including a gradient check with a `2e-4` relative-error bound. - **CLI** with `train`, `sample`, `eval`, `info`, and `next` subcommands. ## Install ```bash git clone https://github.com/prabeakarve/microgpt.git cd microgpt python -m pip install -e . ``` Python 3.10+ and NumPy are the only requirements. ## Quick start Train on the bundled corpus (about 8 KB of text, ~60 seconds on a laptop): ```bash