io

fastvideo.eval.io.NoAudioStreamError

NoAudioStreamError()

Raised when a video file has no audio stream to extract.

fastvideo.eval.io.extract_audio_track

extract_audio_track(video_path: str | Path, *, output_dir: str | Path, sample_rate: int | None = None, codec: str = 'pcm_s16le') -> Path

Pull the first audio stream from video_path to a .wav in output_dir.

Idempotent — if output_dir / {stem}.wav already exists, the existing file is returned without re-decoding. This makes the helper safe to call repeatedly from parallel workers on overlapping inputs (the cache hit is the fast path).

Parameters

video_path : Source video file (anything PyAV can open: mp4, mkv, webm, …). output_dir : Directory the .wav lands in. Created if it doesn't exist. sample_rate : If set, resample to this rate. Default (None) preserves the source rate — most audio metrics resample again internally to their own target rate, so adding a resample step here would just be wasted work. codec : PCM codec for the output. pcm_s16le (default) gives 16-bit little-endian PCM, the most broadly compatible WAV format.

Returns

Path Absolute path to the extracted .wav.

Raises

NoAudioStreamError If video_path has no audio stream.

fastvideo.eval.io.as_video

as_video(x: str | Path | torch.Tensor | Video) -> Video

Coerce path/tensor/Video → :class:Video for the pool to decode.

Path strings and :class:pathlib.Path become Video(source=str(x)); the pool then calls :func:load_video on first use. Tensors become Video(source=None, frames=x) — the pool sees .frames already populated and forwards untouched. :class:Video instances pass through.

fastvideo.eval.io.samples_from

samples_from(*, video: PathSpec | None = None, reference: PathSpec | None = None, audio: PathSpec | None = None, reference_audio: PathSpec | None = None, text_prompt: str | None = None, text_prompts: str | Path | list[str] | None = None, fps: float | None = None, auxiliary_info: dict | list[dict] | None = None, extras: dict | list[dict] | None = None, extract_audio: bool | str | Path = False, extract_workers: int = 4) -> list[dict]

Build a samples list from path-style inputs.

Parameters

video, reference, audio, reference_audio : File path, directory of files (sorted by name), or any iterable of paths. Pass whichever modalities apply to the metrics you plan to run — they attach to sample["video"] / sample["reference"] / sample["audio"] / sample["reference_audio"] respectively. Video paths are wrapped in :class:Video so :class:VideoPool decodes them lazily in parallel; audio paths stay as strings (audio metrics each load with their own resample / preprocess). text_prompt : A single prompt string broadcast onto every sample. text_prompts : A list of strings (one per sample), or a path to a .jsonl / .json file containing per-sample prompts. fps : Scalar fps broadcast onto every sample. auxiliary_info : Single dict (broadcast) or list of dicts (zipped) for sample["auxiliary_info"] — vbench structured-prompt metrics read this. extras : Catch-all per-sample attachments. Use for metric-specific keys the dedicated kwargs don't cover (scenario, view, actions, calibration, reference_take2, ...). Pass a single dict to broadcast or a list-of-dicts to zip; the keys merge into each sample dict. extract_audio : If truthy, auto-extract audio from each video / reference source into .wav files via PyAV and attach the paths under sample["audio"] / sample["reference_audio"]. Pass a path for a persistent cache, True for a tempdir. Skipped silently for videos with no audio stream; ignored wherever audio / reference_audio is already explicit. extract_workers : Parallel workers for extract_audio.

Returns

list[dict] Canonical samples shape — hand directly to :meth:Evaluator.evaluate.

Cardinality and shape

Let N = len(generated inputs) (the agreed length of whichever of video / audio you passed). References are attached 1:1 onto the first N samples; any extras (when |ref| > N) become standalone role-tagged samples at the end of the list, so set metrics like FVD see the full reference corpus while per-sample paired metrics like LPIPS only run on the first N pairs.

Notes

"Missing" keys are simply absent from the sample dict. Metrics handle them per their own contract (sample.get(...) for optional, sample[...] raises for required, or :meth:BaseMetric._skip for opt-in skip behavior). One fat samples list with many keys can serve many metrics — each reads its subset.

fastvideo.eval.io.samples_from_manifest

samples_from_manifest(manifest: str | Path, *, extract_audio: bool | str | Path = False, extract_workers: int = 4) -> list[dict]

Load canonical eval samples from a JSON or JSONL manifest.

Each row may contain video, audio, reference, reference_audio, reference_audio_source, text_prompt and arbitrary metric metadata. reference_audio_source is a media file whose audio track is extracted when extract_audio is enabled. Paths are resolved relative to the manifest directory. Video paths are wrapped as :class:Video handles for lazy decoding; all other keys are preserved. This is the recommended interface for benchmark generation, where matching by sample id is safer than independently sorting folders.

fastvideo.eval.io.build_eval_kwargs

build_eval_kwargs(row: dict, video_path: Path, *, fps: float = 24.0) -> dict[str, Any]

Build evaluator kwargs from a sample row + a video on disk.

Loads the video as (T,C,H,W) and adds the leading batch dim. Forwards prompt (as scalar text_prompt) and auxiliary_info (as scalar dict) when present on the row — matches the one-sample-per-call contract that the evaluator and every metric assume.

fastvideo.eval.io.default_filename

default_filename(row: dict, idx: int, ext: str = '.mp4') -> str

<sanitized-prompt>-<idx>.mp4 — VBench-style.

fastvideo.eval.io.glob_videos

glob_videos(videos_dir: Path, row: dict, ext: str = '.mp4') -> list[Path]

Find every generated video for row, sorted by trailing -<idx>.

fastvideo.eval.io.sanitize_prompt

sanitize_prompt(prompt: str, max_len: int = 100) -> str

Prompt → safe filename stem.

fastvideo.eval.io.extract_frames

extract_frames(video: torch.Tensor, n_frames: int | None = None) -> torch.Tensor

Uniformly sample n_frames from a (T, C, H, W) video tensor.

fastvideo.eval.io.load_video

load_video(source: str | torch.Tensor | list, **kwargs) -> torch.Tensor

Load a video as a (T, C, H, W) float32 tensor in [0, 1].

Supported source types:

• str / Path – path to .mp4 / .avi / .gif file, or a directory of frame images (sorted alphabetically).
• torch.Tensor – returned as-is after shape validation.
• list[PIL.Image] – stacked into a tensor.