🧠 Finetuning¶

This guide covers finetuning video diffusion models with FastVideo, including full finetuning and LoRA.

Training Arguments¶

FastVideo training scripts use several argument groups:

Training Arguments¶

Parallelism Arguments¶

Optimizer Arguments¶

Validation Arguments¶

Full Finetuning¶

Full finetuning updates all model weights. This provides the best quality but requires more GPU memory.

# Example: Wan2.1 T2V 1.3B full finetune (4 GPUs)
bash examples/training/finetune/wan_t2v_1.3B/crush_smol/finetune_t2v.sh

Typical settings:

• Learning rate: 1e-5 to 5e-5
• Gradient checkpointing: --enable_gradient_checkpointing_type "full"
• Memory scaling: Increase --sp_size or reduce --num_latent_t to fit in memory

Attention Quantization-Aware Training¶

Attn-QAT fine-tunes a model while simulating low-bit attention in the forward and backward passes. The modular trainer can select the backend per model role, so a later DMD2 stage can keep fake quantization on the student while the teacher and critic use Flash Attention.

The ready-to-run Wan2.1 MixKit workflow includes supervised fine-tuning, checkpoint export, and three-step DMD2 distillation:

→ Follow the Attn-QAT training guide

LoRA Finetuning¶

LoRA (Low-Rank Adaptation) trains lightweight adapters while keeping the base model frozen. This significantly reduces memory usage and training time.

LoRA-Specific Arguments¶

Learning Rate for LoRA¶

Important: LoRA typically requires a 10–20× higher learning rate than full finetuning because only the low-rank adapters are being trained while the base model is frozen.

Example LoRA Training¶

# Example: Wan2.1 T2V 1.3B LoRA finetune (1 GPU)
bash examples/training/finetune/wan_t2v_1.3B/crush_smol/finetune_t2v_lora.sh

Key differences from full finetune:

• Add --lora_training True --lora_rank 32
• Use higher learning rate (10–20× full finetune)
• Can run on fewer GPUs (even single GPU)
• Outputs adapter weights instead of full model

LoRA Extraction and Merging¶

FastVideo provides tools to extract LoRA adapters from finetuned models and merge them back.

Extract LoRA Adapter¶

Extract a LoRA adapter by comparing a finetuned model to its base:

python scripts/lora_extraction/extract_lora.py \
  --base Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
  --ft path/to/your/finetuned_model \
  --out adapter_r32.safetensors \
  --rank 32

Merge LoRA Adapter¶

Merge an adapter back into a base model:

python scripts/lora_extraction/merge_lora.py \
  --base Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
  --adapter adapter_r32.safetensors \
  --ft path/to/your/finetuned_model \
  --output merged_model

Validate Merged Model¶

Compare the merged model against the original finetuned model:

python scripts/lora_extraction/lora_inference_comparison.py \
  --base merged_model \
  --ft path/to/your/finetuned_model \
  --adapter NONE \
  --output-dir results \
  --prompt "A cat sitting on a windowsill" \
  --compute-ssim \
  --compute-lpips

Training Examples¶

Ready-to-run training scripts are available for multiple models:

→ Browse all training examples

Each example includes:

• download_dataset.sh — download sample data
• preprocess_*.sh — run preprocessing
• finetune_*.sh — full finetune launcher
• finetune_*_lora.sh — LoRA finetune launcher
• validation.json — validation prompts