jwang771/3d-world-simulator

选用指南

AI 根据 README 整理 · 2026/11/3 13:06:51

使用方式

通过命令行进行训练和推理，包括训练变分自编码器、导出潜变量和训练动力学模型。

查看 README 依据

## Training Training has three steps. Each task's datasets, and parameters related to grid and workspace are in `tasks.py`. ### 1. Hybrid particle-voxel autoencoder ```bash python scripts/train_vae.py --task plug --data-root data --out-dir runs/plug/vae ```

`--task` is one of `cloth`, `box`, `pusht`, `plug`, `marker`, `rope`; each task has specific set parameters in `tasks.py`, including specific voxel grid dimensions, workspace bounds, whether it has tactile sensing, among other information. We also include the ability to place object-specific weights during training, which improves the simulation fidelity. ### 2. Latents Dynamics model training is much faster after precomputing latents. To precompute the latents, run the following: ```bash python scripts/export_latents.py --vae-ckpt runs/plug/vae/checkpoints/latest.pth \

--packed-dir data/bimanual_plug_insertion_v3/train --out-dir runs/plug/latents/train python scripts/export_latents.py --vae-ckpt runs/plug/vae/checkpoints/latest.pth \ --packed-dir data/bimanual_plug_insertion_v3/val --out-dir runs/plug/latents/val ``` ### 3. 3D Latent Dynamics ```bash L=runs/plug/latents python scripts/train_dynamics.py --latent-dir $L/train --val-latent-dir $L/val --out-dir runs/plug/dynamics \ --n-history 8 --ar-schedule 1 2 3 4 5 6 7 8 9 10 --ar-schedule-epochs 50 100 150 200 250 300 350 400 450 500 ```

部署要求

需要创建conda环境并安装依赖，数据需按指定格式组织。

查看 README 依据

¹Columbia University, ²UC Berkeley, ³University of Michigan, ⁴NVIDIA, ⁵Analog Devices [Project website](https://jwang771.github.io/3dworldsimulator/) | [Paper](https://jwang771.github.io/3dworldsimulator/paper.pdf) **We will upload data, checkpoints, and inference code soon.** ## Environment ```bash conda env create -f environment.yml conda activate 3dws ``` ## Data Put each dataset under one root, for example `data/bimanual_plug_insertion_v3/{train,val}`. The file layout is in [docs/data_format.md](docs/data_format.md).

已知限制

作者尚未上传数据、检查点和推理代码，目前仅提供训练代码。

查看 README 依据

¹Columbia University, ²UC Berkeley, ³University of Michigan, ⁴NVIDIA, ⁵Analog Devices [Project website](https://jwang771.github.io/3dworldsimulator/) | [Paper](https://jwang771.github.io/3dworldsimulator/paper.pdf) **We will upload data, checkpoints, and inference code soon.** ## Environment ```bash conda env create -f environment.yml conda activate 3dws ``` ## Data Put each dataset under one root, for example `data/bimanual_plug_insertion_v3/{train,val}`. The file layout is in [docs/data_format.md](docs/data_format.md).