apple-aiml-research/ml-halo

选用指南

AI 根据 README 整理 · 2026/11/6 01:06:48

使用方式

通过命令行运行数据生成、训练和部署脚本，支持模拟和真实机器人。

查看 README 依据

| [`system/`](system/) | Closed-loop system: `run_sim.py` / `run_real.py` launchers, generator / tracker / orchestrator nodes, live web viewer; `system/comm` (`halo_comm` messaging) and `system/serve` (`halo_serve`, on-device model scheduling) | [README](system/README.md) | | [`assets/`](assets/) | Unitree G1 robot description (MJCF / URDF / USD), scene objects and scenes | [robots](assets/robots/README.md), [objects](assets/objects/README.md) | ## Data generation To generate the training data, rendered episodes for the generator and motion clips for the tracker, see

- **Tracker on its own (MuJoCo sim-to-sim, real robot):** [`tracker/deploy/README.md`](tracker/deploy/README.md) (Python) and [`tracker/deploy_cpp/README.md`](tracker/deploy_cpp/README.md) (real-time C++) - **Closed-loop in sim (MuJoCo):** [`system/README.md`](system/README.md#run-sim) (`run_sim.py`) - **Closed-loop on the real robot (G1):** [`system/README.md`](system/README.md#run-real) (`run_real.py`) ## Citation If you find this work useful, please cite: ```bibtex @article{halo2026, title = {Perceptive Humanoid Motion from Simulation},

部署要求

需要 Isaac Lab 和 MuJoCo 等仿真环境，以及 Unitree G1 机器人描述文件。

查看 README 依据

 ## Repository layout | Folder | What it is | Docs | |---|---|---| | [`datagen/`](datagen/) | Synthetic data pipeline: Infinigen scenes, Kimodo motion, Isaac / MuJoCo rendering; training data for the generator (rendered episodes) and the tracker (motion clips) | [README](datagen/README.md) | | [`generator/`](generator/) | Motion generator: training and open-loop evaluation | [README](generator/README.md) |

| [`tracker/`](tracker/) | G1 motion-tracking policy: Isaac Lab training, Python and C++ (real-time) deployment | [README](tracker/README.md) | | [`orchestrator/`](orchestrator/) | `halo_orchestrator`: the orchestrator (language request + egocentric image → plan of subtasks) on a local model co-served with the generator | [README](orchestrator/README.md) |

| [`system/`](system/) | Closed-loop system: `run_sim.py` / `run_real.py` launchers, generator / tracker / orchestrator nodes, live web viewer; `system/comm` (`halo_comm` messaging) and `system/serve` (`halo_serve`, on-device model scheduling) | [README](system/README.md) | | [`assets/`](assets/) | Unitree G1 robot description (MJCF / URDF / USD), scene objects and scenes | [robots](assets/robots/README.md), [objects](assets/objects/README.md) | ## Data generation To generate the training data, rendered episodes for the generator and motion clips for the tracker, see