FastH3 Omni Goes Local: DGX Spark, RTX and Apple Silicon

Yesterday we released FastH3 Omni: give it a prompt plus images, video clips and a voice, and it returns a finished video with synchronized sound. On eight GB200s, a 5-second 768p clip from one reference photo takes 5.7 seconds.

Today it comes to your desk. We ran the exact same requests, with identical prompts, references and seeds, end to end on one and two NVIDIA DGX Sparks, an RTX 5090, an RTX 3090 and an Apple M5 Max (64 GB).

TL;DR

• DGX Spark runs every reference mix. A 5-second 480p clip from a photo takes about 3 minutes on one Spark and about 2 minutes on two, and the 128 GB of unified memory handles everything we tested, from one photo to three images or a long reference video, at 480p and 768p.
• Across local hardware. The same 1-photo request takes 62 s on an RTX 5090, about 17 minutes on an RTX 3090 (plus model loading), and 19.5 minutes on an Apple M5 Max (single run) with FastVideo’s new MLX runtime.
• 4.6× faster in seven weeks. Text-to-video on the Spark went from 243 s at our first local release to 52.6 s today. FastH3 Omni runs on the same stack, so those gains carry over.
• References are the real cost. A reference photo makes a request 3.3–4.4× slower than text-to-video on the same machine. We show why, and how to trade some of it back.

Same request, every machine

Choose a reference mix and press Play together. Every panel uses the same prompt, references and seed; outputs differ across hardware, so compare quality rather than pixels.

References 1 imageImage + voice3 images1 short clip1 long clip Next example → 832×480 · 5 s · seed 0

Play together Pause all Listen to DGX Spark · 36L decoderDGX Spark · 8L decoderApple M5 Max · 36L decoderMute all

Playback starts when you press play. Choose one soundtrack with Listen to.

1 image · 5 s request

124 frames · 24 FPS · stereo audio · same prompt, references and seed on every device

Download MP4

Download MP4

Download MP4

Source references (1)

Full prompt

subject_definitions:
<Subject 1> is the young man with dark curly hair wearing a white T-shirt in <Picture 1>.

summary:
[reference generation] <Subject 1> makes breakfast in a sunlit kitchen and speaks warmly to the camera.

retention_analysis:
<Subject 1> (appears in [Shot 1]): fully_preserved - the young man's face, dark curly hair and white T-shirt are retained.

detailed_description:
The target video is in a realistic live-action style with warm morning light.
[Shot 1] A medium close-up with a slow push-in. In a sunlit kitchen, <Subject 1> (S1) sets a bowl of oranges on the wooden counter, wipes his hands on a striped towel, looks into the camera and says with a relaxed smile, <d>[English] Breakfast is on me today, so take your time.</d> Morning light falls across his shoulders and the steam of a kettle drifts behind him.

overall_soundscape:
Bright kitchen room tone, a kettle beginning to whistle, and birdsong drifting through an open window.

non_diegetic_music:
N/A

832×480 · 24 FPS · stereo audio · seed 0. Spark times are three-seed means; the Mac time is a single run.

End-to-end latency

Every bar is the full request, from prompt and references to a finished MP4 with audio.

DGX Spark, every reference mix (three-seed means, 36-layer decoder, 5-second clips):

The two-Spark runs used the earlier FastH3 Omni QAD checkpoint, so read the 1× vs 2× ratio as indicative.

Across local hardware (480p, 36-layer decoder):

The M5 Max numbers are single runs; the RTX 3090 numbers exclude its ~7-minute model load per request.

• Spark runs every mix in minutes, at both resolutions and with either decoder. The 3-image and long-clip requests hit the memory limits of today’s 24–32 GB GPU profiles; the M5 Max runs them, but in 44–59 minutes.
• Two Sparks are 1.5–1.9× faster than one, and the long-clip request scales best (1.9× at 480p).
• The decoder no longer matters much. The 8-layer decoder saves only a few percent on Spark, because the NVFP4 decoders take seconds and the transformer dominates.
• Every class of local machine works. The RTX 5090 is the quickest single GPU at about a minute, and the RTX 3090 and the Apple M5 Max land within 15–20 minutes.

From text-to-video to references

Text-to-video on local hardware has become fast. On the Spark, where FastH3 first went local, the same 5-second 480p clip is 4.6× faster on one Spark and 6.8× faster on two than seven weeks ago (8-layer decoder; the first release was measured cold), on top of the 8.4× we measured over the public vLLM-Omni recipe at launch:

The gains came from eight-step distillation in NVFP4, NVIDIA’s 4-bit floating-point format, then quantization-aware distillation with 4-bit attention and NVFP4 decoders, which cut decoding from 43 s to under 2 s.

Adding a reference photo to the same request takes 3.3–4.4× more time on the same machine:

The reason is tokens. A 5-second 480p clip is about 14,400 video tokens. FastH3 Omni reads each reference image on the 2048-pixel canvas it was trained with, which adds about 7,100 VAE tokens and another 7,100 vision tokens in the prompt. One photo doubles the sequence to about 29,000 tokens, and three photos roughly quadruple it to nearly 60,000. This is why the 3-image request is the slowest everywhere.

The reference canvas is therefore the biggest lever you control. --reference-max-pixels match shrinks each reference to the clip’s own size: on the M5 Max, the 1-photo request drops from 19.5 to 4.7 minutes (4.2×). It is opt-in because it departs from training and changes framing and action:

How it runs on each machine

FastH3 Omni pairs a ~35B-parameter transformer with a ~32B-parameter Qwen3-VL text encoder, well over 100 GB in BF16. Each machine fits it differently.

DGX Spark

Everything stays resident in the Spark’s 128 GB of unified memory: the 4-bit NVFP4 transformer, text encoder and decoder, with no offloading between stages. Attention uses the 4-bit sparse attention on 128-token tiles the model was trained with, while the text, reference and audio rows run densely at 8-bit so the conditioning stays out of FP4 rounding.

Two directly connected Sparks split the token sequence: each holds the full model and processes half the tokens. Conditioning does not split as cleanly as denoising, one reason the speedup stays below 2×.

Get started with the Spark guide or the Spark pair guide.

RTX GPUs

The RTX 5090 keeps the NVFP4 transformer on its 32 GB. A precomputed 155 MB table replaces 26 GB of timestep-projection weights, shrinking the transformer from 44.5 GB to 18.5 GB, and the text encoder loads only while the prompt is encoded.

The RTX 3090 has no FP4 or FP8 hardware, so it runs FP8 weights with BF16 math through the 24 GB profile, streaming the transformer a few blocks at a time.

Apple Silicon

FastH3 Omni runs natively on Macs through FastVideo’s MLX runtime:

• INT6 transformer (17 GB). Timestep projections are precomputed, as on the RTX 5090, and INT6 fits the rest in memory; Apple GPUs have no FP4 units.
• No base VAE needed. The LightVAE Lite encoder (406 MB) reads the references, and the same NVFP4 decoders as on Blackwell write the frames.
• The NVFP4 text encoder runs unchanged, including its vision tower.
• Trained sparsity on Metal, through a new Metal kernel for the model’s 128-token sparse-attention tiles.

Peak memory on the M5 Max stays between 27 and 31 GB across all five workloads.

Quantization-aware distillation

Rounding a distilled eight-step model to 4 bits after training costs quality. Quantization-aware distillation (QAD) instead trains the NVFP4 student against the full-precision teacher with FP4 numerics active from the first step, so the weights learn to absorb the rounding. The sparse-attention gates and norms stay frozen, because retraining them changes which tiles are selected.

FastH3 Omni’s local weights, FastVideo/FastH3-Omni-NVFP4-QAD, extend the FastH3-QAD-v2 recipe to 4-bit attention. The shared training recipe is available in FastVideo #1989.

Try it

Start with the FastVideo cookbook and the FastH3 Omni recipe builder.

Prefer a visual workflow? The local community has been making great things with our ComfyUI release, available in NVFP4 QAD and FP8.

From the community

Within a day of release, people were running FastH3 Omni on their own GPUs:

Community-reported, each on its own setup and settings. Scroll for more; click a post to open it.

What’s next

We will keep pushing what FastH3 Omni can do on Apple Silicon, keep improving quality and speed across every machine, and release smaller models that fit a wider range of consumer hardware.

Limitations

• NVFP4, FP8 and INT6 are different numerical variants, so outputs differ from the BF16 release and from each other for the same seed.
• The M5 Max numbers are single runs, the RTX 3090 numbers exclude model loading, and the two-Spark runs used the earlier QAD checkpoint.
• --reference-max-pixels match trades reference detail for speed.

Acknowledgements

FastH3 builds on MiniMax H3. We thank the MiniMax team for releasing its weights and code.

We thank the NVIDIA Enterprise Products team (Pengcheng Li and Cliff Woolley) for the Video Sparse Attention kernel used by FastH3. We thank the NVIDIA FastGen team (Julius Berner, Chao Liu and Arash Vahdat) for the DMD2 framework and H3 reference work, and for introducing Parallel Decoding Distillation.

We thank Nuva Lab for bringing production-grounded creative video workflows to FastH3, and the upstream serving, kernel and open-source communities whose implementations make these experiments possible. We also acknowledge Reactor for supporting FastH3 Omni, and the vLLM project, NVIDIA and MBZUAI for their support of FastVideo.

Finally, thanks to community member @posi_posi8, who volunteered an Apple M5 Max to benchmark FastH3 Omni for this post.

FastVideo team

Contributor: Aryan Kumar
Tech lead: Will Lin
Advisor: Hao Zhang