DiffuSpace/Clover-35B-A3B-Preview

选用指南

AI 根据 README 整理 · 2026/10/29 00:31:23

使用方式

可通过 Hugging Face Transformers 进行 Python 推理，或使用 SGLang 和 llama.cpp 进行服务与本地推理。

查看 README 依据

[](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks) See the [launch blog](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks) for benchmark results, evaluation settings, and comparisons. ## Quick Start ### Hugging Face Transformers Install PyTorch with CUDA support, then the Hugging Face dependencies. This example uses Transformers 5.19.0 and PyTorch 2.10 or later. ```bash pip install "transformers==5.19.0" accelerate safetensors ```

| Backend | Use case | Decoding | | --- | --- | --- | | [Hugging Face Transformers](#hugging-face-transformers) | Standalone Python inference and experimentation | Block diffusion denoising | | [SGLang](https://github.com/sgl-project/sglang/pull/46927) | Serving and evaluation | Block diffusion denoising and Self-speculative decoding | | [llama.cpp](#sglang-and-llamacpp) | Local inference | Self-speculative decoding | ### SGLang and llama.cpp

部署要求

需要 PyTorch 2.10 或更高版本，以及支持 CUDA 的 GPU，BF16 权重约需 69 GB 显存。

查看 README 依据

[](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks) See the [launch blog](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks) for benchmark results, evaluation settings, and comparisons. ## Quick Start ### Hugging Face Transformers Install PyTorch with CUDA support, then the Hugging Face dependencies. This example uses Transformers 5.19.0 and PyTorch 2.10 or later. ```bash pip install "transformers==5.19.0" accelerate safetensors ```

The example runs on one GPU with enough memory for the BF16 weights (approximately 69 GB) plus inference state. ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Diffuspace/Clover-35B-A3B-Preview" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( model_id, trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda:0", ).eval() messages = [{"role": "user", "content": "What is diffusion?"}] prompt = tokenizer.apply_chat_template( messages,

已知限制

SFT 代码和训练说明尚未发布，计划在 sft/ 目录中提供。

查看 README 依据

Our SGLang implementation supports both **Block diffusion denoising** and **Self-speculative decoding** for Clover. The llama.cpp implementation supports **Self-speculative decoding**. Backend code and setup guides will be added to this repository. > [!NOTE] > We recommend our official **SGLang implementation for serving and evaluation**. Small numerical differences between the Hugging Face implementation and SGLang's kernels can lead to different generated text. ## Supervised Fine-Tuning **Coming soon.** SFT code and training instructions will be released in [`sft/`](./sft/).