---
slug: inference/optimizations
title: Optimizations
---
This page describes the various options for speeding up generation times in FastVideo.
:::note[On a DGX Spark (GB10)?]
Several options on this page behave differently on the GB10's unified-memory
hardware — some give little or nothing there. See
[DGX Spark: Performance & Tuning](/FastVideo/getting_started/installation/spark_performance)
for what actually helps on that platform and why. Two Sparks, one clip:
[Pair two NVIDIA DGX Sparks](/FastVideo/getting_started/installation/spark_pair).
:::
## Table of Contents [#table-of-contents]
- Optimized Attention Backends
- [Flash Attention](#flash-attention)
- [Sliding Tile Attention (Archived)](#sliding-tile-attention-archived)
- [Sage Attention](#sage-attention)
- [Sage Attention 3](#sage-attention-3)
- [FP8 Weight Quantization](#fp8-weight-quantization)
- [Adaptive Guidance (CFG gating)](#adaptive-guidance-cfg-gating)
- [torch.compile](#torch-compile)
## Attention Backends [#attention-backends]
### Available Backends [#available-backends]
- Torch SDPA: `FASTVIDEO_ATTENTION_BACKEND=TORCH_SDPA`
- Flash Attention 2 and 3: `FASTVIDEO_ATTENTION_BACKEND=FLASH_ATTN`
- Video Sparse Attention: `FASTVIDEO_ATTENTION_BACKEND=VIDEO_SPARSE_ATTN`
- Sage Attention: `FASTVIDEO_ATTENTION_BACKEND=SAGE_ATTN`
- Sage Attention 3: `FASTVIDEO_ATTENTION_BACKEND=SAGE_ATTN_THREE`
- Attn-QAT inference (modified SageAttention3 FP4, sm_120a/sm_121a — RTX 5090 / DGX Spark): `FASTVIDEO_ATTENTION_BACKEND=ATTN_QAT_INFER`
- Video MoBA Attention: `FASTVIDEO_ATTENTION_BACKEND=VMOBA_ATTN`
- Sparse Linear Attention: `FASTVIDEO_ATTENTION_BACKEND=SLA_ATTN`
- SageSLA Attention: `FASTVIDEO_ATTENTION_BACKEND=SAGE_SLA_ATTN`
- Sliding Tile Attention (archived branch only):
`FASTVIDEO_ATTENTION_BACKEND=SLIDING_TILE_ATTN`
### Configuring Backends [#configuring-backends]
There are two ways to configure the attention backend in FastVideo.
#### 1. In Python [#1-in-python]
In python, set the `FASTVIDEO_ATTENTION_BACKEND` environment variable before instantiating `VideoGenerator` like this:
```python
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "VIDEO_SPARSE_ATTN"
```
#### 2. In CLI [#2-in-cli]
You can also set the environment variable on the command line:
```bash
FASTVIDEO_ATTENTION_BACKEND=SAGE_ATTN python example.py
```
### Flash Attention [#flash-attention]
**`FLASH_ATTN`**
We recommend always installing [Flash Attention 2](https://github.com/Dao-AILab/flash-attention):
```bash
uv pip install flash-attn==2.7.4.post1 --no-build-isolation
```
And if using a Hopper+ GPU (ie H100), installing [Flash Attention 3](https://github.com/Dao-AILab/flash-attention?tab=readme-ov-file#flashattention-3-beta-release) by compiling it from source (takes about 10 minutes for me):
```bash
git clone https://github.com/Dao-AILab/flash-attention.git && cd flash-attention
cd hopper
uv pip install ninja
python setup.py install
```
### Flash Attention 4 (opt-in) [#flash-attention-4-opt-in]
FastVideo never auto-selects FlashAttention-4 (`flash_attn.cute`) just because it
is installed: its CuTeDSL kernels JIT-compile per shape family and can fail at
runtime on some GPU/shape combinations. To use FA4, install the pinned
`flash-attn-4` build (see the `flash-attn-4` source in `pyproject.toml`) and set:
```bash
UV_TORCH_BACKEND=cu130 uv pip install -e ".[fasth3]"
export FASTVIDEO_FA4=1
```
On GPUs below sm90 a capability gate routes to FlashAttention-2 the calls FA4
cannot serve there: grad-enabled (training) attention (FA4's backward requires
sm90+) and GQA attention (FA4's `pack_gqa` fails to JIT-compile below sm90).
On sm90+ both run on FA4. If FA4 is unusable while `FASTVIDEO_FA4=1` is set,
FastVideo fails loudly instead of silently falling back.
MiniMax-H3 can additionally use FA4's packed-varlen entry point for its long,
single-sequence dense DiT self-attention:
```bash
export FASTVIDEO_FA4=1
export FASTVIDEO_MINIMAX_H3_FA4_PACKED_VARLEN=1
```
This route is inference-only and remains disabled by default. Runtime guards
keep masked attention, batch sizes above one, unequal query/key lengths,
grad-enabled calls, and NVFP4 on their established paths. It does not apply to
the Preview checkpoint's sparse VSA blocks. Packed-varlen changes floating-point
reduction order relative to fixed-length FA4, so treat it as a speed/quality
evaluation option rather than an exact-parity mode.
### FP4 Flash Attention 4 (Blackwell only) [#fp4-flash-attention-4-blackwell-only]
**`FLASH_ATTN`** with **`--nvfp4_fa4`**
On Blackwell GPUs (B200/B300), you can enable FP4 quantized Q/K attention for up to **1.31x kernel speedup** over BF16 FA4, peaking at **2018 TFLOPS**. This quantizes Q and K to NVFP4 E2M1 with per-block E4M3 scale factors while keeping V in BF16 or FP8.
See the [Attn-QAT paper](https://arxiv.org/abs/2603.00040) and [flash-attention-fp4 benchmark results](https://github.com/hao-ai-lab/flash-attention-fp4/blob/fp4/flash_attn/cute/README.md) for details.
#### Requirements [#requirements]
- **GPU**: NVIDIA Blackwell (sm100a or sm103a) — B200, B300, GB200, GB300
- **CUDA**: 13.0+
- **Python**: 3.10 or 3.11
#### Installation [#installation]
Install the FP4 flash attention kernel (without upgrading your existing torch):
```bash
# branch fix/cutlass-dsl-4.5 carries the cutlass-dsl 4.5 fix (cute.core.ThrMma
# -> cute.ThrMma); switch back to @fp4 once hao-ai-lab/flash-attention-fp4#2 merges.
pip install --no-deps "git+ssh://git@github.com/hao-ai-lab/flash-attention-fp4.git@fix/cutlass-dsl-4.5#subdirectory=flash_attn/cute"
pip install "nvidia-cutlass-dsl>=4.5.2" apache-tvm-ffi flashinfer-python
```
The `--no-deps` flag prevents upgrading torch/torchvision. Use the supported
PyTorch 2.12.0 and CUDA 13 environment for this kernel.
Branch-to-`nvidia-cutlass-dsl` compatibility (the fork tracks the CuTe DSL API
surface closely):
| fork branch | cutlass-dsl | notes |
|---|---|---|
| `fp4` | `==4.4.2` (+ `nvidia-cutlass-dsl-libs-base==4.4.2`) | validated set on GB200: `quack-kernels==0.4.1`, `flashinfer-python==0.6.8`, `CUTE_DSL_ENABLE_TVM_FFI=1`, `FASTVIDEO_FA4=1` |
| `fix/cutlass-dsl-4.5` | `>=4.5.2` | carries the `cute.core.ThrMma` -> `cute.ThrMma` fix |
| any | 4.6-era | unsupported: `cute.make_fragment` was removed at module level; fails at CuTe JIT trace |
`FASTVIDEO_FA4=1` is required alongside the fork: it ships no compiled
FlashAttention-2, so dense attention paths raise ImportError without the FA4
opt-in. The same kernel also serves `ATTN_QAT_INFER` on sm_100a/sm_103a
(datacenter Blackwell) — the selection log's receipt line
(`ATTN_QAT_INFER resolved: ...`) records the arch, kernel, and quantization
mode that actually bound.
#### Usage [#usage]
Enable FP4 attention via the `--nvfp4_fa4` flag:
```bash
python examples/inference/optimizations/fp4_attn_wan2_1_1_3b.py --nvfp4_fa4
```
Or in Python via the `nvfp4_fa4` kwarg (sets env vars automatically):
```python
from fastvideo import VideoGenerator
gen = VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
nvfp4_fa4=True,
num_gpus=1,
use_fsdp_inference=False, # FSDP is incompatible with FP4 pointer path
)
gen.generate_video(prompt="A raccoon in sunflowers", save_video=True)
```
#### fp8 PV mode **[target]** [#fp8-pv-mode-target]
The FA4-FP4 path keeps V in BF16 by default. The `fa4_pv_mode` knob (an
attention-impl field accepted by both `FLASH_ATTN` and `ATTN_QAT_INFER` on this
path; allowed values `"bf16"`/`"fp8"`) casts V to fp8 e4m3 before the kernel —
an unscaled cast, per the kernel's plain-fp8 PV contract. Model code constructs
attention with fixed literals, so the user-reachable switch is the
`FASTVIDEO_FA4_PV_MODE` env var (passing `fa4_pv_mode="fp8"` to
`VideoGenerator.from_pretrained` sets it for you):
```bash
FASTVIDEO_FA4_PV_MODE=fp8 python examples/inference/optimizations/fp4_attn_wan2_1_1_3b.py --nvfp4_fa4
```
Kernel-level benchmarks on datacenter Blackwell show it is faster at large
shapes, but it stays opt-in **[target]** pending an end-to-end compiled
benchmark and a quality gate before any default consideration. The
`ATTN_QAT_INFER` receipt line reports the configured `pv_mode`, and the V dtype
actually fed to the kernel is logged once on the first FA4 forward.
#### Known Limitations [#known-limitations]
- `use_fsdp_inference=True` is incompatible with the FP4 path (FSDP shards invalidate tensor pointers)
- Per-call cosine similarity vs BF16: ~0.99 (slight quantization error accumulates over denoising steps)
- Only supports `headdim >= 128`
### NVFP4 Attn-QAT attention (modified SageAttention3, Blackwell sm_120a/sm_121a) [#nvfp4-attn-qat-attention-modified-sageattention3-blackwell-sm_120asm_121a]
**`ATTN_QAT_INFER`**
Quantizes Q/K/V inside the modified SageAttention3 FP4 attention backend. On
DGX Spark, keep the transformer linear layers in BF16: the separate
`transformer_quant=nvfp4_qat` linear path is not supported on sm_121 yet.
The `attn_qat_infer` kernel targets consumer/workstation Blackwell — **sm_120a
(RTX 5090 / PRO 6000)** and **sm_121a (DGX Spark GB10)**; the block-scaled FP4
MMA is numerically correct on both (GB10-verified: cos ~0.98 vs bf16 SDPA).
sm_121a support needs a CUDA 13 build (`TORCH_CUDA_ARCH_LIST=12.1a`). A runtime
capability gate falls back to Flash Attention on other GPUs.
For VSA-distilled MiniMax-H3 students, the same kernel has a block-sparse
forward that runs VSA's 64-token tile selection in FP4
(`FASTVIDEO_H3_VSA_FP4=1`). See
[FastH3 NVFP4 on RTX PRO 6000](/FastVideo/inference/fasth3_rtx_pro_6000).
> **Quality note (stock weights):** on stock Wan-2.1 the FP4-attention output is
> below bf16 — QAD expects a QAT-distilled checkpoint the model was *trained* to
> tolerate FP4 attention with. Post-hoc on non-QAT weights, expect a quality
> cost; use the QAT checkpoint for release-grade output.
See the [Attn-QAT paper](https://arxiv.org/abs/2603.00040).
Enable FP4 attention with BF16 transformer linears on DGX Spark:
```python
import os
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "ATTN_QAT_INFER"
from fastvideo import VideoGenerator
gen = VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
num_gpus=1,
use_fsdp_inference=False,
)
gen.generate(request={"prompt": "A raccoon in sunflowers", "output": {"save_video": True}})
```
The full QAD configuration, pairing this attention backend with
`transformer_quant=nvfp4_qat`, is currently limited to sm_120. Its RTX 5090
example is:
```bash
python examples/inference/optimizations/nvfp4_qat_wan2_1_1_3b.py
python examples/inference/optimizations/nvfp4_qat_wan2_1_1_3b.py --bf16 # baseline
```
### Sliding Tile Attention (Archived) [#sliding-tile-attention-archived]
**`SLIDING_TILE_ATTN`**
The full STA integration in `fastvideo/` is archived from `main` and preserved
at:
- https://github.com/hao-ai-lab/FastVideo/tree/sta_do_not_delete
We keep STA off `main` because we believe VSA is strictly better than STA for
the actively maintained FastVideo path.
Kernel code in `fastvideo-kernel` is still retained. For mask search and STA
inference workflow, see [STA docs](/FastVideo/attention/sta).
### Video Sparse Attention [#video-sparse-attention]
**`VIDEO_SPARSE_ATTN`**
Video Sparse Attention is provided by `fastvideo-kernel`.
See [VSA docs](/FastVideo/attention/vsa) for installation details.
### Sage Attention [#sage-attention]
**`SAGE_ATTN`**
To use [SageAttention](https://github.com/thu-ml/SageAttention) 2.1.1, please compile from source:
```bash
git clone https://github.com/thu-ml/SageAttention.git
cd sageattention
python setup.py install # or uv pip install -e .
```
### Sage Attention 3 [#sage-attention-3]
**`SAGE_ATTN_THREE`**
[SageAttention 3](https://github.com/thu-ml/SageAttention/tree/main/sageattention3_blackwell) is an advanced attention mechanism that leverages FP4 quantization and Blackwell GPU Tensor Cores for significant performance improvements.
#### Hardware Requirements [#hardware-requirements]
- RTX5090
#### Installation [#installation_1]
Note that Sage Attention 3 requires `python>=3.13`, `torch>=2.8.0`, and CUDA 13.
If you are using `uv` and `torch==2.8.0`, make sure that
`sentencepiece==0.2.1` in the `pyproject.toml` file.
To use Sage Attention 3 in FastVideo, follow the `README.md` in the linked repository to install the package from source.
### V-MoBA / SLA / SageSLA [#v-moba-sla-sagesla]
These backends are model-specific and require the corresponding kernels and
dependencies. Use the support matrix and model examples to confirm compatibility
before enabling them.
## FP8 Weight Quantization [#fp8-weight-quantization]
**`transformer_quant="FP8"`**
Quantizes DiT linear layers (attention projections and FFN) to FP8 e4m3.
On GPUs older than sm89, the FP8 matmul falls back to a bf16 dequant path
automatically.
### Requirements [#requirements_1]
- **GPU**: sm89+ (H100, L40S, RTX 4090, or newer) for hardware FP8 compute
- **ROCm**: CDNA4 (MI350X / MI355X, gfx950) runs the FP8 `_scaled_mm` path through
hipBLASLt (OCP e4m3fn). MI300X (gfx942) only exposes the `fnuz` FP8 formats and
takes the bf16 dequant fallback like a pre-sm89 GPU.
- No additional packages required beyond the base FastVideo install
### Usage [#usage_1]
```python
from fastvideo import VideoGenerator
from fastvideo.layers.quantization import get_quantization_config
gen = VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
# Pass an instance — the bare string is not resolved on the from_pretrained path.
transformer_quant=get_quantization_config("FP8")(), # per-tensor (default)
# transformer_quant=get_quantization_config("FP8")(granularity="channel"), # slower, higher accuracy
)
gen.generate(request={"prompt": "A raccoon in sunflowers", "output": {"save_video": True}})
```
Or run the example script:
```bash
python examples/inference/optimizations/fp8_wan2_1_1_3b.py
python examples/inference/optimizations/fp8_wan2_1_1_3b.py --granularity channel
python examples/inference/optimizations/fp8_wan2_1_1_3b.py --bf16 # baseline
```
### Granularity [#granularity]
| Mode | Weight scales | Activation scales | Speed | Accuracy |
|------|--------------|-------------------|-------|----------|
| `tensor` (default) | per-tensor | per-tensor | faster | lower |
| `channel` | per-output-channel | per-token (rowwise) | slower | higher |
## torch.compile [#torchcompile]
FastVideo can `torch.compile` the DiT (transformer) for a substantial
end-to-end speedup. It is **off by default** and enabled per-run.
### Enabling [#enabling]
```python
generator = VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
enable_torch_compile=True,
)
```
A complete A/B example (eager vs compiled, warmup excluded) is in
[`examples/inference/optimizations/torch_compile_example.py`](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/optimizations/torch_compile_example.py).
`fastvideo generate` is config-file driven; to enable `torch.compile`
from the CLI, set the relevant field in your run config and pass it via
`fastvideo generate --config run.yaml`. There is no top-level
`--enable-torch-compile` flag on the subcommand.
Only DiT submodules that declare `_compile_conditions` are compiled
(most shipped models). The text encoder and VAE are not compiled by this
flag.
### Regional fullgraph compile (experimental) [#regional-fullgraph-compile-experimental]
`inference_torch_compile` is a stricter, kwargs-free variant that ports the
training-side regional compile of
[#1718](https://github.com/hao-ai-lab/FastVideo/pull/1718) to inference: the
loader wraps each `_compile_conditions` block in
`torch.compile(fullgraph=True)` with inductor
`options={"emulate_precision_casts": True}` right after the transformer
loads. The ordinary compile path keeps its historical compiler-disabled
attention boundary by default; regional compile opts in only the compatible
attention instances owned by this transformer. MiniMax-H3 VSA is supported
only by the inference-only ptx_blackwell tile-64 route
(`FASTVIDEO_VSA_PTX_BLACKWELL=1` and `VSA_tile_size=64`); the loader probes that
route before capture and keeps the transformer eager when the kernel or
device is unsupported. Legacy VSA, MiniMax-H3 tile-256 VSA, and the explicit
`FASTVIDEO_DISABLE_ATTENTION_COMPILE=1` escape hatch keep the transformer
eager with one warning instead of failing mid-denoise.
```python
generator = VideoGenerator.from_pretrained(
"MiniMaxAI/MiniMax-H3",
inference_torch_compile=True, # or FASTVIDEO_INFERENCE_TORCH_COMPILE=1
)
```
Do not combine it with `torch_compile_kwargs['mode']` (the loader injects
inductor options, and torch.compile forbids mode+options); it is
independent of `enable_torch_compile`, and when both are set the regional
compile wins for the DiT.
### What to expect from generic compile [#what-to-expect-from-generic-compile]
The Wan result below measures the existing generic
`enable_torch_compile=True` path. It is useful evidence that compile can help,
but it is **not** a benchmark or numerical gate for the stricter regional
fullgraph path above.
| Config | Effect |
|---|---|
| Wan2.1-T2V-1.3B, A100-80GB, 480×832×81f, 50 steps | end-to-end **259.7s → 198.1s (−23.7%)**; per-step **4.91 → 3.78 s/it** |
The speedup is **configuration-dependent** — it varies with model,
resolution, step count, and GPU. Treat the number above as one measured
data point, not a guarantee; benchmark your own config (recipe below).
There is a **one-time graph-build cost** on the first generation (tens of
seconds to minutes, model-dependent). It amortizes over subsequent
generations with the same input shapes. Always exclude the first
(warmup) generation when measuring steady-state latency — measuring the
warmup is the most common way to wrongly conclude "compile is slower".
**Regional MiniMax-H3 accuracy caveat (job 2660).** On one GB200 at
768×1344×124, the native 50-point schedule ran exactly 49 transformer
forwards. After one warmup, three fixed-prompt/fixed-seed repeats averaged
**185.08s → 157.01s end to end** and **174.90s → 147.12s denoising**. Each
leg was independently pixel-deterministic, but compiled output did **not**
match eager: mean absolute pixel error **20.247/255**, PSNR **16.67 dB**,
mean SSIM **0.7108**, and mean MS-SSIM **0.6370** across 124 frames. Treat
regional MiniMax-H3 compile as an opt-in performance experiment, not an
eager-parity-safe mode.
The same caveat applies to sparse MiniMax-H3 regional compile. Its mask
compaction, ptx_blackwell launch, trained compression gates, and inference-only H3
fusions are fullgraph-compatible, but compilation can still change model
numerics. The FastH3 `all` profile enables regional compile by default because
it is the fastest measured route at 124, 243, and 345 frames. Use
`--no-inference-torch-compile` when comparing against the eager sparse-DiT
route.
**Numerics.** Inductor's lowering is designed to preserve eager
semantics within floating-point tolerance, but per-model equivalence is
not asserted by any standing SSIM regression here — the SSIM tests in
[`fastvideo/tests/ssim/`](https://github.com/hao-ai-lab/FastVideo/tree/main/fastvideo/tests/ssim)
run with `enable_torch_compile` disabled. If you depend on compile
output staying close to eager (or your previous compiled run), run an
MS-SSIM gate on *your* config, especially when combining
`enable_torch_compile=True` with other numerics-affecting flags
(quantized attention backends, FP4, layerwise offload edge cases).
### Known interactions [#known-interactions]
- **Layerwise CPU offload** (`dit_layerwise_offload=True`, the default):
the offload hook previously caused an implicit graph break once per
transformer layer, fragmenting the compiled region. Addressed in
hao-ai-lab/FastVideo#1365 — keep that fix to get a clean compiled
region under the default offload path.
- **`mode="reduce-overhead"` / CUDA graphs**: not yet supported
by regional compile because that path injects inductor `options`, and
PyTorch rejects `mode` together with `options`. The generic compile path
can accept `mode`, but CUDA-graph compatibility remains backend- and
shape-dependent. FA2/FA3 inference and FA4 expose traceable custom-op
boundaries. MiniMax-H3's ptx_blackwell tile-64 inference route is the only VSA
path in the regional support envelope; other VSA paths and the FA3
grad-enabled path remain outside it. Use the default inductor mode shown
above unless your exact configuration has its own gate.
Extra `torch.compile` options are passed through `torch_compile_kwargs`
(a dict), accepted by `VideoGenerator.from_pretrained(...)` and by the
CLI as a JSON string via `--torch-compile-kwargs`. Example (currently
**not** recommended — see the CUDA-graphs caveat above):
```python
VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
enable_torch_compile=True,
torch_compile_kwargs={"mode": "reduce-overhead"}, # may error today
)
```
### Benchmarking torch.compile [#benchmarking-torchcompile]
Same discipline as attention backends — same prompt, same seed, same
config; **discard the first generation** (graph build):
```python
import time
from fastvideo import VideoGenerator
gen = VideoGenerator.from_pretrained("your-model-id", enable_torch_compile=True)
req = {"prompt": "Your prompt", "sampling": {"seed": 1024},
"output": {"save_video": False}}
gen.generate(req) # warmup: graph build, discard
t0 = time.perf_counter()
gen.generate(req) # measured: shapes reused
print(f"compiled steady-state: {time.perf_counter() - t0:.2f}s")
```
See `examples/inference/optimizations/torch_compile_example.py` for a
baseline-vs-compile A/B with the warmup correctly excluded.
## Benchmarking different optimizations [#benchmarking-different-optimizations]
To benchmark backend performance, generate the same prompt with the same seed and compare end-to-end generation times:
```python
import os
import time
for backend in ["TORCH_SDPA", "FLASH_ATTN", "SAGE_ATTN"]:
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = backend
generator = VideoGenerator.from_pretrained("your-model-id")
start_time = time.perf_counter()
generator.generate_video(
prompt="Your prompt",
seed=1024,
)
elapsed = time.perf_counter() - start_time
print(f"{backend}: {elapsed:.2f}s")
```
Note: reinstantiate `VideoGenerator` after changing `FASTVIDEO_ATTENTION_BACKEND`.
## Adaptive Guidance (CFG gating) [#adaptive-guidance-cfg-gating]
CFG gating accelerates classifier-free guidance by reusing the cached
`noise_pred_cond - noise_pred_uncond` delta after a configurable fraction of
the denoising schedule, skipping the unconditional model forward for the
remaining steps. The technique is the LinearAG variant of Adaptive Guidance
(Castillo et al. 2023, [arXiv:2312.12487](https://arxiv.org/abs/2312.12487)).
### Enabling [#enabling_1]
Set the `FASTVIDEO_CFG_GATE_STEP` environment variable to a float in `[0, 1]`:
| Value | Behavior |
|-------|----------|
| `1.0` (default) | Disabled — legacy two-pass CFG every step. |
| `0.5` | Cache the delta after `len(timesteps) * 0.5` steps; reuse for the rest. |
| `0.0` | Cache from the very first step (most aggressive). |
```bash
export FASTVIDEO_CFG_GATE_STEP=0.5
```
### Trade-offs [#trade-offs]
- **Memory**: one extra model-output-sized tensor per rank held during the
gating window.
- **Quality**: VBench-measured quality is preserved within noise on 4 of 5
dimensions at `FASTVIDEO_CFG_GATE_STEP=0.5` for Wan T2V 1.3B per the PR's
reported numbers (see [#1372](https://github.com/hao-ai-lab/FastVideo/pull/1372)).
- **Speed**: ~22% e2e on 4xL40S and ~24% on 1xH100 at the same settings.
Default behavior is byte-for-byte equivalent to the legacy two-pass CFG path;
the feature is fully opt-in.