---
slug: contributing/dreamverse-development
title: Dreamverse Development
---
<a id="dreamverse-development"></a>

Dreamverse lives under `apps/dreamverse/` as a product app inside the
FastVideo monorepo. Backend code uses the local FastVideo workspace package;
frontend tooling remains standalone under `apps/dreamverse/web/`.

## Backend tests {#backend-tests}

Run backend tests excluding GPU-marked cases from the FastVideo repository
root:

```bash
uv run --locked --package dreamverse --extra test pytest apps/dreamverse/dreamverse/tests/ -m 'not gpu' -q
```

Collection can still touch FastVideo streaming imports that probe for an active
GPU driver, so the corresponding CI job runs on a GPU even with GPU-marked tests
excluded.

## Backend launch {#backend-launch}

Launch the migrated backend through the installed console commands:

```bash
dreamverse-server --port 8009
dreamverse-mock-server --port 8009
```

If `dreamverse-server` is missing, install FastVideo with the `dreamverse`
extra from the checkout:

```bash
UV_TORCH_BACKEND=cu126 uv pip install -e ".[dreamverse]"  # use cu130 on CUDA 13
```

## Frontend build and tests {#frontend-build-and-tests}

Run frontend commands from the standalone web app:

```bash
cd apps/dreamverse/web
npm ci
npm run build
npm test
```

Playwright is intentionally run against a live backend as part of the local GPU
manual verification flow, not in the Phase 3 migration gate.

## Local GPU verification {#local-gpu-verification}

Choose an available physical GPU for full-stack smoke tests. For example,
`CUDA_VISIBLE_DEVICES=4` makes physical GPU 4 appear as logical GPU 0 inside
the process.

For a managed backend-and-frontend redeploy with readiness checks and logs,
use the repo-local skill helper:

```bash
./.agents/skills/dreamverse-deploy/scripts/dreamverse-deploy.sh 4 8009 5299
```

The helper's legacy frontend default is `5274`, so the example passes the web
app's current port, `5299`, explicitly. It writes logs under
`/tmp/opencode/dreamverse-deploy`. The equivalent manual backend launch is:

```bash
CUDA_VISIBLE_DEVICES=4 dreamverse-server --host 0.0.0.0 --port 8009
```

In another shell, verify the service:

```bash
curl -s http://localhost:8009/healthz
```

The full Playwright suite expects `/healthz`, `/readyz`, `/status`,
`/prompt-system-config`, and `/curated-presets` from the Dreamverse backend.

## H3 generation modes {#h3-generation-modes}

Dreamverse exposes the running model's supported modes through
`GET /generation-capabilities`. The default LTX profiles and FastH3 Preview
support T2VA. Start the full checkpoint profile to enable all three modes:

```bash
DREAMVERSE_MODEL_ID=full-h3 DREAMVERSE_SP_SIZE=4 FASTVIDEO_GPU_COUNT=4 \
  dreamverse-server --host 127.0.0.1 --port 8009
```

This profile uses the full `MiniMaxAI/MiniMax-H3` weights, 50 denoising steps,
and four GPUs on one node by default. It does not apply the T2VA-only Preview
LoRA. On Slurm, run inside your allocation and preserve the scheduler's
`CUDA_VISIBLE_DEVICES`; the local single-GPU deployment helper is not suitable.
See the [Slurm demo launcher](https://github.com/hao-ai-lab/FastVideo/blob/main/apps/dreamverse/scripts/slurm/README.md).

| Mode | Inputs | H3 pipeline |
| --- | --- | --- |
| T2VA | Text prompt; no conditioning assets | Base |
| FL2VA | First-frame image required, last-frame image optional | Base with first/last-frame conditioning |
| Ref2VA | Ordered image/video/audio references, at least one visual reference | Reference transformer |

The Asset List owns uploads and the composer selects their roles. Ref2VA allows
at most nine image, three video, three audio, and twelve total references.
Images accept PNG, JPEG, and WebP up to 15 MiB and 16 megapixels. Videos accept
MP4, MOV, and WebM; audio accepts WAV, MP3, M4A, FLAC, OGG, and WebM. Video/audio
uploads are limited to 100 MiB and 30 seconds, and videos must be 4K or smaller.
Ref2VA images must have aspect ratios between 1:4 and 4:1. Audio, including
video soundtracks, must be mono or stereo.
The runtime validates actual media content with Pillow or FFprobe. FFmpeg and
FFprobe must be installed; PyAV is also required by the H3 reference pipeline.

Uploads use `POST /assets` with a raw file body, a supported `Content-Type`, and
an optional percent-encoded `X-Asset-Name`. The response includes an opaque
`asset_id`, media metadata, and a same-origin preview URL. GET/HEAD and DELETE
are supported at `/assets/{asset_id}`. The runtime library is limited to 100
assets and 2 GiB. Assets remain available until deleted or the runtime exits;
project metadata persists in the browser, but expired uploads must be uploaded
again after a runtime restart. Assets in an active session cannot be deleted.

Both `session_init_v2` and `project_init_v1` accept the same typed fields:

```json
{
  "generation_mode": "fl2va",
  "conditioning_assets": [
    {"asset_id": "<first-image-id>", "role": "first_frame"},
    {"asset_id": "<last-image-id>", "role": "last_frame"}
  ]
}
```

Ref2VA entries use `role: "reference"`; their array order is meaningful. T2VA
uses an empty array. Do not send file paths or binary media inside websocket
JSON. Omitting both fields preserves the legacy protocol, including its
`initial_image` support. Explicit modes cannot also use `initial_image`.
Invalid requests receive `error_code: "invalid_generation_input"` before GPU
generation. The mode and assets are locked within a project.

FL2VA applies the selected first/last frames to the first segment; later
segments continue from the previous final frame. Ref2VA reuses its ordered
references for each segment, producing reference-conditioned clips without
promising temporal continuation. Starting a project in a different H3 pipeline
unloads the previous executor before loading the other transformer. This can
take time; both transformers are never intentionally kept resident together.
Full H3 sessions default to a two-hour limit, configurable with
`DREAMVERSE_SESSION_TIMEOUT_SECONDS`.

![Ref2VA input selection in the explicitly labeled mock demo](https://github.com/hao-ai-lab/FastVideo/blob/main/assets/images/dreamverse-ref2va-demo.png)

### Mode validation and demo {#mode-validation-and-demo}

GPU-independent upload, parser, and mock-stream contract tests can run with a
minimal Python environment (FastAPI, Pillow, HTTPX, NumPy, and pytest):

```bash
python -m pytest apps/dreamverse/dreamverse/tests/test_generation_inputs.py \
  apps/dreamverse/dreamverse/tests/test_mock_server.py -q
```

For UI development, start `dreamverse-mock-server --port 8009`, then the normal
frontend. The page explicitly labels this as a mock demo. It validates real
uploads and mode contracts but returns a synthetic test-pattern clip; it does
not test H3 output quality or GPU performance. Run the generation-mode
Playwright spec against that server and use an actual `full-h3` runtime to
validate model output. Real acceptance requires a completed audio/video clip
for each mode, plus a base-to-reference and reference-to-base project switch.

## Production-equivalent GPU prerequisites {#production-equivalent-gpu-prerequisites}

For the production-equivalent NVFP4 path, install these dependencies
in the FastVideo `.venv` before GPU smoke tests:

```bash
uv pip install --python .venv/bin/python \
  flashinfer-python flash-attn cerebras-cloud-sdk openai \
  --no-build-isolation
```

| Package | Why |
|---|---|
| `flashinfer-python` | Required for NVFP4 quantization. Without it, model load fails with `ImportError: NVFP4 quantization requires flashinfer`. |
| `flash-attn` | Optional but recommended; without it attention falls back to Torch SDPA (functional but slower). |
| `cerebras-cloud-sdk` | Required by the migrated prompt enhancer for the default `cerebras` provider. |
| `openai` | Required by the prompt enhancer's OpenAI-compatible providers + downstream rewrites. |

### B200 / sm_100a + gcc-15 conda toolchain (flashinfer JIT workaround) {#b200-sm_100a-gcc-15-conda-toolchain-flashinfer-jit-workaround}

On hosts where the conda toolchain ships gcc-15 (which nvcc rejects with
`#error -- unsupported GNU version! gcc versions later than 14 are not
supported!`), set these env vars before launching anything that triggers
flashinfer's JIT kernel build:

```bash
export CC=/usr/bin/gcc-13
export CXX=/usr/bin/g++-13
export CUDAHOSTCXX=/usr/bin/g++-13
export NVCC_PREPEND_FLAGS="-ccbin /usr/bin/gcc-13 -allow-unsupported-compiler"
```

`dreamverse-server` does not set these; keep them in the launching shell when
starting it directly. The repo-local `dreamverse-deploy` skill exports them for
managed local launches.