NVIDIA DGX Spark (GB10)¶

Instructions to install FastVideo on an NVIDIA DGX Spark — the GB10 Grace-Blackwell platform.

The Spark is ARM64 (aarch64) with CUDA 13, which the standard NVIDIA GPU guide does not cover: there is no prebuilt aarch64 wheel for fastvideo-kernel, so it is compiled from source as part of the install, and the system Python typically lacks the development headers that build needs. The steps below handle both, without requiring sudo.

Requirements¶

OS: Linux (aarch64)
GPU: NVIDIA GB10 (compute capability sm_121), at least 1
CUDA Toolkit: 13.x at /usr/local/cuda (nvcc on PATH)
Python: 3.10–3.12
Compilers: gcc/g++ 10+ (the Spark ships gcc 13)

Different compute capability?

These instructions target the GB10's sm_121. The kernel build auto-detects it from your GPU, so there is normally nothing to set. To check your device:

python -c "import torch; print(torch.cuda.get_device_capability(0))"  # GB10 -> (12, 1)

Install¶

Run from the repository root:

# 0. (once) install uv and put it on PATH
curl -LsSf https://astral.sh/uv/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"
git clone https://github.com/hao-ai-lab/FastVideo.git && cd FastVideo

# 1. Create a venv on a uv-managed CPython 3.12 (bundles the dev headers the
#    kernel build needs; the Spark's system python3.12 lacks them).
uv venv .venv --python 3.12 --python-preference only-managed --seed
source .venv/bin/activate

# 2. Initialise the kernel submodules (cutlass + ThunderKittens headers).
git submodule update --init --recursive \
  fastvideo-kernel/include/cutlass fastvideo-kernel/include/tk

# 3. Install FastVideo (editable; compiles the in-tree kernel for the GB10).
UV_TORCH_BACKEND=cu130 uv pip install -e .

Contributors who want the lint/test tooling: UV_TORCH_BACKEND=cu130 uv pip install -e ".[dev]".

Then jump to Verify the install.

Building without a visible GPU (CI / Docker)¶

With no GPU visible the kernel build can't probe the arch and auto can't detect the driver — name both explicitly:

UV_TORCH_BACKEND=cu130 TORCH_CUDA_ARCH_LIST=12.1 uv pip install -e .

Verify the install¶

python - <<'PY'
import torch, fastvideo
print("fastvideo:", fastvideo.__version__)
print("torch    :", torch.__version__, "| cuda:", torch.version.cuda, "| avail:", torch.cuda.is_available())
print("device   :", torch.cuda.get_device_name(0))
PY

fastvideo --help

Expected: a +cu130 torch, device: NVIDIA GB10, and the CLI listing generate / serve / router-serve / bench / eval.

To confirm the compiled CUDA kernel actually runs on the GB10 (importing alone doesn't execute the .so):

python - <<'PY'
import torch
from fastvideo_kernel import Int8Linear
lin  = torch.nn.Linear(512, 256, bias=False).cuda().to(torch.bfloat16)
# .cuda() is required: from_linear() leaves the int8 buffers on the CPU
qlin = Int8Linear.from_linear(lin, quantize=True).cuda()  # compiled quant_cuda
x    = torch.randn(128, 512, device="cuda", dtype=torch.bfloat16)
y, ref = qlin(x), lin(x)                                  # compiled gemm_cuda
rel = (y.float() - ref.float()).norm() / ref.float().norm()
print(f"int8 GEMM rel err vs fp32: {rel.item():.4f}  (~0.01 is correct; int8 is lossy)")
PY

Manual build (advanced / fallback)¶

The one-liner above is the supported path. Build the kernel yourself only if you are iterating on the CUDA source or the auto-build fails:

# install torch + the kernel's build deps into the active venv first
UV_TORCH_BACKEND=cu130 uv pip install torch torchvision torchaudio scikit-build-core cmake ninja setuptools wheel
# build just the kernel against that torch (auto-detects sm_121)
uv pip install ./fastvideo-kernel --no-build-isolation
# then the rest of FastVideo
uv pip install -e .

Or cd fastvideo-kernel && ./build.sh, which auto-detects the arch and does the same compile (it initializes only the kernel's cutlass and tk submodules).

Optional: flash-attn¶

Not installed by default. FastVideo falls back to other attention backends without it. For the guide's Python 3.12, PyTorch 2.12, and CUDA 13 environment, install the matching prebuilt Linux ARM64 wheel from the FlashAttention prebuilt wheel index:

uv pip install "https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.9.22/flash_attn-2.8.3%2Bcu130torch2.12-cp312-cp312-linux_aarch64.whl"

Troubleshooting¶

Symptom	Cause / Fix
`Could NOT find Python (missing: ... Development.Module)`	venv built from system Python without headers. Recreate with `--python-preference only-managed` (add `--clear` to reuse the path), or `sudo apt install python3.12-dev`.
kernel build can't find cutlass headers	Submodules not initialised — run the `git submodule update` step.
`fastvideo-kernel: could not determine the target CUDA architecture`	The build couldn't see a GPU and no arch was given. Build on the Spark itself, or pass `TORCH_CUDA_ARCH_LIST=12.1` (see Building without a visible GPU).
`nvcc fatal: Unsupported gpu architecture 'compute_121'`	`nvcc` older than CUDA 12.9/13. Confirm `nvcc --version` is 13.x and `CUDACXX=/usr/local/cuda/bin/nvcc`.
`ninja: command not found` (manual build only)	`uv pip install scikit-build-core cmake ninja setuptools wheel`.

If you hit other issues, please open an issue on our GitHub repository. You can also join our Slack community for additional support.

Development Environment Setup¶

If you're planning to contribute to FastVideo please see the Contributor Guide.