---
slug: training/examples/minimax-h3_synthetic
title: MiniMax H3 Ref2VA synthetic training fixture
---
**Source:** [examples/training/finetune/minimax-h3/synthetic](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/training/finetune/minimax-h3/synthetic)

<a id="minimax-h3-ref2va-synthetic-training-fixture"></a>

This directory demonstrates the raw manifest accepted by the MiniMax H3
Ref2VA preprocessor. The checked-in media is deterministic, repository-owned
synthetic content: a geometric animation, a still geometric reference, and
mathematically generated tones. It contains no people or third-party media.

## Raw manifest contract {#raw-manifest-contract}

The preprocessor accepts JSON, JSONL, or a JSON object containing a `data`
array. Every sample has this shape:

```json
{
  "schema_version": "minimax_h3_ref2va_raw_v1",
  "id": "unique-sample-id",
  "target": {"video": "relative/or/absolute/target.mp4"},
  "caption": "The target description.",
  "references": [
    {"type": "image", "image": "reference.png"},
    {"type": "video", "video": "silent-reference.mp4"},
    {"type": "video_audio", "video": "reference-with-sound.mp4", "audio": "reference-with-sound.mp4"},
    {"type": "audio", "audio": "reference.wav"}
  ]
}
```

Relative media paths resolve from the manifest directory. `video` is always a
silent visual reference even when its container has an audio stream;
`video_audio` explicitly uses both streams and requires both `video` and
`audio` fields. When both fields name the same container, its own audio stream
is used as the soundtrack. A sample supports at most 12 total
references: 9 images, 3 videos, and 3 standalone audio references. A non-empty
reference list must include at least one image or video. FL2VA training is not
part of this workflow.

Preprocessing encodes target video/audio latents, dynamic Qwen3-VL hidden
states and tags, ordered visual anchors at clean-time `0.999`, and clean audio
anchors. It writes one validated Parquet row per shard. The video VAE, audio
VAE, and Qwen3-VL stack are loaded sequentially on one GPU; plan for the full
MiniMax H3 component checkpoint and substantial preprocessing time even for a
small manifest.

## Fixture generation and provenance {#fixture-generation-and-provenance}

`generate_fixture.py` is the complete source for all three checked-in media
files. It uses fixed formulas and codec settings; it requires NumPy, Pillow,
and PyAV from the FastVideo environment. The visual/audio content is source
reproducible, but MP4/AAC container bytes are not claimed to be bit-identical
across PyAV, FFmpeg, or libx264 versions. `PROVENANCE.json` records the exact
generation library versions used for the checked-in bytes. Regenerate or
verify them with:

```bash
python examples/training/finetune/minimax-h3/synthetic/generate_fixture.py
python examples/training/finetune/minimax-h3/synthetic/generate_fixture.py --verify
```

`--verify` checks the checked-in bytes against the recorded hashes; regeneration
updates both the media and provenance for the active codec environment.
`PROVENANCE.json` records the exact generation parameters and SHA-256 digest of
each media file. The source and generated artifacts are licensed under the
repository's Apache-2.0 license. They have no external source, third-party
rights, likeness, voice, consent, or redistribution dependency.

## Configurable setup {#configurable-setup}

The setup script verifies the checked-in fixture before manifest validation
and accepts all machine-specific inputs through environment variables:

```bash
MODEL_DIR=/path/to/MiniMax-H3 \
MANIFEST=/path/to/train.jsonl \
OUTPUT_DIR=/path/to/preprocessed-ref2va \
PYTHON_BIN=/path/to/python \
CUDA_VISIBLE_DEVICES=0 \
bash examples/train/setup_minimax_h3_ref2va_synthetic_single_sample.sh
```

`ENV_FILE=/path/to/optional.env` may be supplied when the selected checkpoint
requires environment-based authentication. It is never required for local
inputs. Existing non-empty output is refused by default. Set
`REPLACE_EXISTING=1` to stage and validate a replacement; the previous dataset
is moved to a sibling `.backup-*` directory rather than deleted.

The example training configs describe a 64-rank topology with SP=8 and HSDP
8x8. Adjust paths and distributed dimensions to the actual cluster. A modular
training checkpoint is DCP state, not an inference directory. Export a
checkpoint with:

```bash
python -m fastvideo.train.entrypoint.dcp_to_diffusers \
  --checkpoint /path/to/checkpoint-400 \
  --output-dir /path/to/minimax-h3-export \
  --verify
```

Full Ref2VA tuning replaces `transformer_ref/`. H3 LoRA export merges the
adapter into native transformer weights and writes the same component used by
the public T2VA or Ref2VA inference pipeline. Standalone H3 adapter export and
runtime adapter selection are not supported by this workflow.

Export gathers the complete transformer state on CPU while the live training
model still exists. `--verify` releases that training graph before strict
reload, but it does not make the gather streaming. For the roughly 62 GiB H3
transformer, use a host with comfortably more than the combined live-model,
gathered-state, and runtime working set; a 121 GiB unified-memory GB10 is not a
validated full-export target.


## Additional Files {#additional-files}

??? note "PROVENANCE.json"

    ```json
    {
      "copyright": "FastVideo contributors",
      "description": "Deterministic geometric animation and mathematically synthesized tones.",
      "generation_environment": {
        "audio_encoder": "aac",
        "libav": {
          "libavcodec": "62.28.102",
          "libavdevice": "62.3.102",
          "libavfilter": "11.14.102",
          "libavformat": "62.12.102",
          "libavutil": "60.26.102",
          "libswresample": "6.3.102",
          "libswscale": "9.5.102"
        },
        "numpy": "2.5.2",
        "pillow": "11.3.0",
        "pyav": "18.1.0",
        "video_encoder": "libx264"
      },
      "generator": "generate_fixture.py",
      "human_subjects": false,
      "license": "Apache-2.0",
      "parameters": {
        "fps": 24,
        "height": 128,
        "num_frames": 124,
        "sample_rate": 32000,
        "width": 224
      },
      "schema_version": "fastvideo_synthetic_fixture_v1",
      "sha256": {
        "refs/audios/synthetic_tones.wav": "20478230383ea1dc09cb06f84a737d04afc797e93e094d4909427b557e9579f5",
        "refs/images/geometric_reference.png": "3e5a5783e3673c03e9fd14fa6e8b97c7364c5385ae3da3a87fe60a27f8fb04db",
        "targets/geometric_motion.mp4": "058d8a11d151c0959c6ba56c4e56e82aa7a77952f7be35589c5d96458bcf4f71"
      },
      "third_party_content": false
    }
    
    ```

??? note "generate_fixture.py"

    ```py
    # SPDX-License-Identifier: Apache-2.0
    """Generate the repository-owned MiniMax H3 Ref2VA synthetic fixture."""
    
    from __future__ import annotations
    
    import argparse
    import hashlib
    import json
    import math
    from pathlib import Path
    import wave
    
    import av
    import numpy as np
    from PIL import Image, ImageDraw, __version__ as PILLOW_VERSION
    
    ROOT = Path(__file__).resolve().parent
    VIDEO_PATH = ROOT / "targets" / "geometric_motion.mp4"
    IMAGE_PATH = ROOT / "refs" / "images" / "geometric_reference.png"
    AUDIO_PATH = ROOT / "refs" / "audios" / "synthetic_tones.wav"
    PROVENANCE_PATH = ROOT / "PROVENANCE.json"
    
    WIDTH = 224
    HEIGHT = 128
    FPS = 24
    NUM_FRAMES = 124
    SAMPLE_RATE = 32_000
    NUM_SAMPLES = math.ceil(NUM_FRAMES * SAMPLE_RATE / FPS)
    MEDIA_PATHS = (VIDEO_PATH, IMAGE_PATH, AUDIO_PATH)
    
    
    def _pixels(frame_index: int) -> np.ndarray:
        x = np.arange(WIDTH, dtype=np.uint16)[None, :]
        y = np.arange(HEIGHT, dtype=np.uint16)[:, None]
        phase = frame_index * 3
        pixels = np.empty((HEIGHT, WIDTH, 3), dtype=np.uint8)
        pixels[..., 0] = ((x + phase) % 256).astype(np.uint8)
        pixels[..., 1] = ((2 * y + phase) % 256).astype(np.uint8)
        pixels[..., 2] = (((x // 2) + (y // 2) + phase) % 256).astype(np.uint8)
    
        image = Image.fromarray(pixels)
        draw = ImageDraw.Draw(image)
        for grid_x in range(0, WIDTH, 32):
            draw.line((grid_x, 0, grid_x, HEIGHT - 1), fill=(235, 240, 255), width=1)
        for grid_y in range(0, HEIGHT, 32):
            draw.line((0, grid_y, WIDTH - 1, grid_y), fill=(235, 240, 255), width=1)
        center_x = 20 + (frame_index * 3) % (WIDTH - 40)
        center_y = HEIGHT // 2 + round(22 * math.sin(frame_index * 2 * math.pi / NUM_FRAMES))
        draw.ellipse((center_x - 12, center_y - 12, center_x + 12, center_y + 12),
                     fill=(20, 70, 220), outline=(255, 255, 255), width=2)
        return np.asarray(image)
    
    
    def _stereo_samples() -> np.ndarray:
        sample = np.arange(NUM_SAMPLES, dtype=np.float64)
        seconds = sample / SAMPLE_RATE
        phase = 2 * np.pi * (220.0 * seconds + 0.5 * 110.0 * seconds**2 / (NUM_SAMPLES / SAMPLE_RATE))
        left = 0.24 * np.sin(phase) + 0.08 * np.sin(2 * np.pi * 440.0 * seconds)
        right = 0.24 * np.sin(phase + np.pi / 3) + 0.08 * np.sin(2 * np.pi * 330.0 * seconds)
        return np.round(np.stack((left, right), axis=1) * 32767).astype(np.int16)
    
    
    def _write_image() -> None:
        image = Image.fromarray(_pixels(0))
        image.save(IMAGE_PATH, format="PNG", compress_level=9, optimize=False)
    
    
    def _write_audio(samples: np.ndarray) -> None:
        with wave.open(str(AUDIO_PATH), "wb") as output:
            output.setnchannels(2)
            output.setsampwidth(2)
            output.setframerate(SAMPLE_RATE)
            output.writeframes(samples.astype("<i2", copy=False).tobytes())
    
    
    def _write_video(samples: np.ndarray) -> None:
        with av.open(str(VIDEO_PATH), "w") as container:
            container.metadata.clear()
            video_stream = container.add_stream("libx264", rate=FPS, options={"preset": "slow", "crf": "18"})
            video_stream.width = WIDTH
            video_stream.height = HEIGHT
            video_stream.pix_fmt = "yuv420p"
            video_stream.codec_context.thread_count = 1
            audio_stream = container.add_stream("aac", rate=SAMPLE_RATE)
            audio_stream.codec_context.layout = "stereo"
    
            for frame_index in range(NUM_FRAMES):
                frame = av.VideoFrame.from_ndarray(_pixels(frame_index), format="rgb24")
                frame.pts = frame_index
                container.mux(video_stream.encode(frame))
            container.mux(video_stream.encode())
    
            interleaved = samples.reshape(1, -1)
            audio_frame = av.AudioFrame.from_ndarray(interleaved, format="s16", layout="stereo")
            audio_frame.sample_rate = SAMPLE_RATE
            resampler = av.audio.resampler.AudioResampler(format="fltp", layout="stereo", rate=SAMPLE_RATE)
            pts = 0
            for frame in resampler.resample(audio_frame):
                frame.pts = pts
                pts += frame.samples
                container.mux(audio_stream.encode(frame))
            container.mux(audio_stream.encode())
    
    
    def _sha256(path: Path) -> str:
        digest = hashlib.sha256()
        with path.open("rb") as handle:
            for chunk in iter(lambda: handle.read(1024 * 1024), b""):
                digest.update(chunk)
        return digest.hexdigest()
    
    
    def _relative(path: Path) -> str:
        return path.relative_to(ROOT).as_posix()
    
    
    def generate() -> None:
        for path in MEDIA_PATHS:
            path.parent.mkdir(parents=True, exist_ok=True)
        samples = _stereo_samples()
        _write_image()
        _write_audio(samples)
        _write_video(samples)
        provenance = {
            "schema_version": "fastvideo_synthetic_fixture_v1",
            "generator": "generate_fixture.py",
            "license": "Apache-2.0",
            "copyright": "FastVideo contributors",
            "third_party_content": False,
            "human_subjects": False,
            "description": "Deterministic geometric animation and mathematically synthesized tones.",
            "generation_environment": {
                "numpy": np.__version__,
                "pillow": PILLOW_VERSION,
                "pyav": av.__version__,
                "libav": {name: ".".join(str(part) for part in version)
                          for name, version in sorted(av.library_versions.items())},
                "video_encoder": "libx264",
                "audio_encoder": "aac",
            },
            "parameters": {
                "width": WIDTH,
                "height": HEIGHT,
                "fps": FPS,
                "num_frames": NUM_FRAMES,
                "sample_rate": SAMPLE_RATE,
            },
            "sha256": {_relative(path): _sha256(path) for path in MEDIA_PATHS},
        }
        PROVENANCE_PATH.write_text(json.dumps(provenance, indent=2, sort_keys=True) + "\n", encoding="utf-8")
    
    
    def verify() -> None:
        provenance = json.loads(PROVENANCE_PATH.read_text(encoding="utf-8"))
        expected = provenance.get("sha256", {})
        actual = {_relative(path): _sha256(path) for path in MEDIA_PATHS}
        if actual != expected:
            raise SystemExit(f"Synthetic fixture hash mismatch:\nexpected={expected}\nactual={actual}")
        print(json.dumps(actual, indent=2, sort_keys=True))
    
    
    def main() -> None:
        parser = argparse.ArgumentParser(description=__doc__)
        parser.add_argument("--verify", action="store_true", help="Verify checked-in media against PROVENANCE.json")
        args = parser.parse_args()
        if args.verify:
            verify()
        else:
            generate()
            verify()
    
    
    if __name__ == "__main__":
        main()
    
    ```

??? note "train.jsonl"

    ```jsonl
    {"schema_version":"minimax_h3_ref2va_raw_v1","id":"synthetic_geometric_train","target":{"video":"targets/geometric_motion.mp4"},"caption":"A blue circle moves across a colorful geometric grid while synthetic tones sweep gently in pitch.","references":[{"type":"image","image":"refs/images/geometric_reference.png"},{"type":"audio","audio":"refs/audios/synthetic_tones.wav"}]}
    
    ```

??? note "validation.json"

    ```json
    {
      "data": [
        {
          "schema_version": "minimax_h3_ref2va_raw_v1",
          "id": "synthetic_geometric_validation",
          "target": {"video": "targets/geometric_motion.mp4"},
          "caption": "A blue circle moves across a colorful geometric grid while synthetic tones sweep gently in pitch.",
          "references": [
            {"type": "image", "image": "refs/images/geometric_reference.png"},
            {"type": "audio", "audio": "refs/audios/synthetic_tones.wav"}
          ]
        }
      ]
    }
    
    ```