---
slug: inference/configuration
title: Configuration
---
<a id="configuration"></a>

## Multi-GPU Setup {#multi-gpu-setup}

FastVideo automatically distributes the generation process when multiple GPUs are specified:

```python
# Will use 4 GPUs in parallel for faster generation
generator = VideoGenerator.from_pretrained(
    "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
    num_gpus=4,
)
```

One node uses the multiprocessing executor (`execution_backend: mp`, the
default). With `num_gpus=1`, `mp` automatically selects the in-process (`uni`)
executor so weights load in the current process. Set `execution_backend: uni`
to force that path. Two machines — for example two DGX Sparks, one GPU each —
need Ray:

```yaml
generator:
  engine:
    num_gpus: 2
    execution_backend: ray
    parallelism:
      sp_size: 2
```

Set `RAY_ADDRESS` and `FASTVIDEO_HOST_IP` to the interconnect IPs, not Wi-Fi.
The FastH3 example selects Ray automatically when `RAY_ADDRESS` is set. Full
bring-up: [Pair two NVIDIA DGX Sparks](/FastVideo/getting_started/installation/spark_pair).

## Customizing Generation {#customizing-generation}

- `PipelineConfig`: Initialization time parameters
- `SamplingParam`: Generation time parameters

You can customize generation behavior using `PipelineConfig` and
`SamplingParam`:

```python
from fastvideo import VideoGenerator, SamplingParam, PipelineConfig

def main():
    model_name = "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
    config = PipelineConfig.from_pretrained(model_name)
    config.vae_precision = "fp16"

    # Create the generator
    generator = VideoGenerator.from_pretrained(
        model_name,
        num_gpus=1,
        dit_layerwise_offload=True,  # FastVideoArgs option
        pipeline_config=config
    )

    # Create and customize sampling parameters
    sampling_param = SamplingParam.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
    
    # How many frames to generate
    sampling_param.num_frames = 45
    
    # Video resolution (width, height)
    sampling_param.width = 1024
    sampling_param.height = 576
    
    # How many steps we denoise the video (higher = better quality, slower generation)
    sampling_param.num_inference_steps = 30
    
    # How strongly the video conforms to the prompt (higher = more faithful to prompt)
    sampling_param.guidance_scale = 7.5
    
    # Random seed for reproducibility
    sampling_param.seed = 42  # Optional, leave unset for random results

    # Generate video with custom parameters
    prompt = "A beautiful sunset over a calm ocean, with gentle waves."
    video = generator.generate_video(
        prompt, 
        sampling_param=sampling_param, 
        output_path="my_videos/",  # Controls where videos are saved
        save_video=True
    )

    # If return_frames=True, frames are available in video["frames"]
    print(f"Generated {len(video['frames'])} frames")

if __name__ == '__main__':
    main()
```

## JSON/YAML Config Files (CLI) {#jsonyaml-config-files-cli}

The inference CLI is config-first. Use an explicit subcommand with `--config`,
then apply optional dotted overrides on top, matching the training CLI style.
By default, CLI generation uses `return_frames=false` unless you set
`request.output.return_frames: true` in config or via a dotted override.

```bash
fastvideo generate --config config.yaml
```

Example nested config:

```yaml
generator:
  model_path: FastVideo/FastHunyuan-diffusers
  engine:
    num_gpus: 2
    parallelism:
      sp_size: 2
request:
  prompt: A capybara relaxing in a hammock
  sampling:
    num_frames: 45
    height: 720
    width: 1280
    num_inference_steps: 6
    seed: 1024
  output:
    output_path: outputs/
```

`generator.engine.execution_backend` accepts `mp`, `uni`, `ray`, or
`external_launcher`. The last option is for synchronized offline generation
under `torchrun` or `srun`; it is not supported by `fastvideo serve`. Set
`generator.engine.num_gpus` and the relevant parallelism sizes to the total
launcher world size. See the [distributed inference contract](/FastVideo/inference/architecture#distributed-inference)
and the [MiniMax-H3 run config](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/minimax_h3_external_launcher.yaml).

`generator.engine.parallelism.dist_timeout` is an optional positive integer
in **seconds**. FastVideo applies it to the default process group when
FastVideo initializes that group, and to every FastVideo-created device and
Gloo control group. If it is omitted, PyTorch's backend defaults apply. This
timeout bounds a missing collective participant; external launchers should
still supervise the full worker group and terminate peers when one rank exits.

Override individual values from the CLI with dotted paths:

```bash
fastvideo generate --config config.yaml --request.sampling.seed 42
```

## Performance Optimization {#performance-optimization}

For configuring optimizations, please see our [optimizations guide](/FastVideo/inference/optimizations)