PSRben/VisionHOPE

选用指南

AI 根据 README 整理 · 2026/10/14 18:16:04

适用场景

适用于ImageNet-1K分类、COCO检测与实例分割、ADE20K语义分割等视觉任务的研究与实验。

查看 README 依据

<a href="#efficiency">Efficiency</a> </p> </div> PyTorch implementation of **VisionHOPE**, a visual backbone based on self-referential nested learning (SRNL). This repository provides ImageNet classification, COCO detection and instance segmentation, and ADE20K semantic segmentation, along with standalone SRNL and VisionHOPE operator modules.

| Task | Sizes | Script name | | --- | --- | --- | | ImageNet-1K | `tiny`, `small`, `base` | `visionhope_<size>.sh` | | COCO, Mask R-CNN 1× | `tiny`, `small`, `base` | `visionhope_<size>_coco_1x.sh` | | COCO, Mask R-CNN 3× | `tiny`, `small` | `visionhope_<size>_coco_3x.sh` | | ADE20K, UPerNet | `tiny`, `small`, `base` | `visionhope_<size>_ade20k.sh` | For example, to train VisionHOPE-S on eight GPUs: ```bash # ImageNet-1K NPROC_PER_NODE=8 bash scripts/train/visionhope_small.sh \ --data-root data/imagenet --output outputs/visionhope_small # COCO

使用方式

通过命令行脚本进行训练和评估，也提供Python API加载模型和独立算子模块。

查看 README 依据

python -m pip install --upgrade huggingface_hub hf download PSRben/VisionHOPE visionhope_small.pth --local-dir weights ``` Replace `visionhope_small.pth` with the checkpoint filename for your model and task. To download all classification, COCO, and ADE20K checkpoints: ```bash hf download PSRben/VisionHOPE --include "visionhope_*.pth" --local-dir weights ``` Load an ImageNet-1K checkpoint with Python: ```python import torch from visionhope.models import create_model model = create_model( "visionhope_small", checkpoint_path="weights/visionhope_small.pth" ).cuda().eval()

with torch.inference_mode(): logits = model(torch.randn(1, 3, 224, 224, device="cuda")) # [1, 1000] ``` Omit `checkpoint_path` to create a model for training. ## SRNL and VisionHOPE operator **SRNL** accepts sequences of shape `[batch, tokens, dim]` and returns the same shape: ```python import torch from visionhope.models import SRNL srnl = SRNL(dim=64, head_dim=16, chunk_size=64).cuda() x = torch.randn(2, 257, 64, device="cuda", requires_grad=True) y = srnl(x) y.square().mean().backward() ```

| Task | Sizes | Script name | | --- | --- | --- | | ImageNet-1K | `tiny`, `small`, `base` | `visionhope_<size>.sh` | | COCO, Mask R-CNN 1× | `tiny`, `small`, `base` | `visionhope_<size>_coco_1x.sh` | | COCO, Mask R-CNN 3× | `tiny`, `small` | `visionhope_<size>_coco_3x.sh` | | ADE20K, UPerNet | `tiny`, `small`, `base` | `visionhope_<size>_ade20k.sh` | For example, to train VisionHOPE-S on eight GPUs: ```bash # ImageNet-1K NPROC_PER_NODE=8 bash scripts/train/visionhope_small.sh \ --data-root data/imagenet --output outputs/visionhope_small # COCO

部署要求

需要Linux系统、NVIDIA GPU和CUDA工具包（含nvcc和C++编译器），推荐Python 3.10，分类任务也支持Python 3.11，需安装CUDA 12.1和PyTorch 2.1.0。

查看 README 依据

The hierarchical **Tiny / Small / Base** backbones support classification and dense prediction. For use in other architectures, see the standalone [SRNL, operator, and block interfaces](#srnl-and-visionhope-operator). ## Installation Use Linux with an NVIDIA GPU and the CUDA toolkit, including `nvcc` and a C++ compiler. Python 3.10 is recommended for all tasks; classification also supports Python 3.11. Run the following from the repository root, with CUDA 12.1 installed: ```bash python -m pip install torch==2.1.0 torchvision==0.16.0 \ --index-url https://download.pytorch.org/whl/cu121

python -m pip install -r requirements/classification.txt python -m pip install --no-deps -e . ``` CUDA extensions compile on first use. Keep model parameters in FP32 and use `torch.autocast` for mixed precision. <details> <summary>Additional dependencies for COCO and ADE20K</summary> Use a separate Python 3.10 environment for each task, with the PyTorch and CUDA versions above. Install MMCV, then the task dependencies: ```bash python -m pip install --only-binary=mmcv mmcv==2.1.0 \ -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.1/index.html ``` For COCO: ```bash

已知限制

所有组件（SRNL、VisionHOPEOperator、VisionHOPEBlock）需要CUDA支持，不支持CPU运行。

查看 README 依据

block = VisionHOPEBlock(dim=64, mixer_dim=32, head_dim=16).cuda() y = operator(x) z = block(x) ``` `mixer_dim` sets the block's internal width. All three components require CUDA and support training with autograd. `chunk_size=None` uses 64-token chunks for standalone SRNL and row/column chunks for image operators and blocks. You can also specify a positive chunk length; rectangular feature maps and sequences with an incomplete final chunk are supported. ## Fast inference For standalone SRNL and operators, set `fast_inference=True`, call `.eval()`, and disable gradients: ```python