---
title: io
---

<div class="python-reference">
<a id="fastvideo.eval.io-classes"></a>
<a id="fastvideo.eval.io-functions"></a>
<a id="fastvideo.eval.io.extract_audio_track--parameters"></a>
<a id="fastvideo.eval.io.extract_audio_track--returns"></a>
<a id="fastvideo.eval.io.extract_audio_track--raises"></a>
<a id="fastvideo.eval.io.samples_from--parameters"></a>
<a id="fastvideo.eval.io.samples_from--returns"></a>
<a id="fastvideo.eval.io.samples_from--cardinality-and-shape"></a>
<a id="fastvideo.eval.io.samples_from--notes"></a>
<a id="fastvideo.eval.io"></a>

<h2 id="fastvideo.eval.io.NoAudioStreamError"><code>fastvideo.eval.io.NoAudioStreamError</code></h2>
<pre class="doc-signature"><code>NoAudioStreamError()</code></pre>
<p>Raised when a video file has no audio stream to extract.</p>
<h2 id="fastvideo.eval.io.extract_audio_track"><code>fastvideo.eval.io.extract_audio_track</code></h2>
<pre class="doc-signature"><code>extract_audio_track(video_path: str | Path, *, output_dir: str | Path, sample_rate: int | None = None, codec: str = &#x27;pcm_s16le&#x27;) -&gt; Path</code></pre>
<p>Pull the first audio stream from <em>video_path</em> to a <code>.wav</code> in <em>output_dir</em>.</p>
<p>Idempotent — if <code>output_dir / &#123;stem&#125;.wav</code> already exists, the
existing file is returned without re-decoding.  This makes the
helper safe to call repeatedly from parallel workers on overlapping
inputs (the cache hit is the fast path).</p>
<h2>Parameters</h2>
<p>video_path :
    Source video file (anything PyAV can open: mp4, mkv, webm, …).
output_dir :
    Directory the <code>.wav</code> lands in.  Created if it doesn't exist.
sample_rate :
    If set, resample to this rate.  Default (<code>None</code>) preserves the
    source rate — most audio metrics resample again internally to
    their own target rate, so adding a resample step here would just
    be wasted work.
codec :
    PCM codec for the output.  <code>pcm_s16le</code> (default) gives 16-bit
    little-endian PCM, the most broadly compatible WAV format.</p>
<h2>Returns</h2>
<p>Path
    Absolute path to the extracted <code>.wav</code>.</p>
<h2>Raises</h2>
<p>NoAudioStreamError
    If <em>video_path</em> has no audio stream.</p>
<h2 id="fastvideo.eval.io.as_video"><code>fastvideo.eval.io.as_video</code></h2>
<pre class="doc-signature"><code>as_video(x: str | Path | torch.Tensor | Video) -&gt; Video</code></pre>
<p>Coerce path/tensor/Video → :class:<code>Video</code> for the pool to decode.</p>
<p>Path strings and :class:<code>pathlib.Path</code> become <code>Video(source=str(x))</code>;
the pool then calls :func:<code>load_video</code> on first use.  Tensors become
<code>Video(source=None, frames=x)</code> — the pool sees <code>.frames</code> already
populated and forwards untouched.  :class:<code>Video</code> instances pass through.</p>
<h2 id="fastvideo.eval.io.samples_from"><code>fastvideo.eval.io.samples_from</code></h2>
<pre class="doc-signature"><code>samples_from(*, video: PathSpec | None = None, reference: PathSpec | None = None, audio: PathSpec | None = None, reference_audio: PathSpec | None = None, text_prompt: str | None = None, text_prompts: str | Path | list&#91;str&#93; | None = None, fps: float | None = None, auxiliary_info: dict | list&#91;dict&#93; | None = None, extras: dict | list&#91;dict&#93; | None = None, extract_audio: bool | str | Path = False, extract_workers: int = 4) -&gt; list&#91;dict&#93;</code></pre>
<p>Build a samples list from path-style inputs.</p>
<h2>Parameters</h2>
<p>video, reference, audio, reference_audio :
    File path, directory of files (sorted by name), or any iterable
    of paths.  Pass whichever modalities apply to the metrics you
    plan to run — they attach to <code>sample["video"]</code> /
    <code>sample["reference"]</code> / <code>sample["audio"]</code> /
    <code>sample["reference_audio"]</code> respectively.  Video paths are
    wrapped in :class:<code>Video</code> so :class:<code>VideoPool</code> decodes them
    lazily in parallel; audio paths stay as strings (audio metrics
    each load with their own resample / preprocess).
text_prompt :
    A single prompt string broadcast onto every sample.
text_prompts :
    A list of strings (one per sample), or a path to a <code>.jsonl</code> /
    <code>.json</code> file containing per-sample prompts.
fps :
    Scalar fps broadcast onto every sample.
auxiliary_info :
    Single dict (broadcast) or list of dicts (zipped) for
    <code>sample["auxiliary_info"]</code> — vbench structured-prompt
    metrics read this.
extras :
    Catch-all per-sample attachments.  Use for metric-specific keys
    the dedicated kwargs don't cover (<code>scenario</code>, <code>view</code>,
    <code>actions</code>, <code>calibration</code>, <code>reference_take2</code>, ...).  Pass
    a single dict to broadcast or a list-of-dicts to zip; the keys
    merge into each sample dict.
extract_audio :
    If truthy, auto-extract audio from each <code>video</code> /
    <code>reference</code> source into <code>.wav</code> files via PyAV and attach
    the paths under <code>sample["audio"]</code> / <code>sample["reference_audio"]</code>.
    Pass a path for a persistent cache, <code>True</code> for a tempdir.
    Skipped silently for videos with no audio stream; ignored
    wherever <code>audio</code> / <code>reference_audio</code> is already explicit.
extract_workers :
    Parallel workers for <code>extract_audio</code>.</p>
<h2>Returns</h2>
<p>list[dict]
    Canonical samples shape — hand directly to
    :meth:<code>Evaluator.evaluate</code>.</p>
<h2>Cardinality and shape</h2>
<p>Let <code>N = len(generated inputs)</code> (the agreed length of whichever
of <code>video</code> / <code>audio</code> you passed).  References are attached
1:1 onto the first N samples; any extras (when <code>|ref| &gt; N</code>)
become standalone role-tagged samples at the end of the list, so
set metrics like FVD see the full reference corpus while per-sample
paired metrics like LPIPS only run on the first N pairs.</p>
<h2>Notes</h2>
<p>"Missing" keys are simply absent from the sample dict.  Metrics
handle them per their own contract (<code>sample.get(...)</code> for
optional, <code>sample[...]</code> raises for required, or
:meth:<code>BaseMetric._skip</code> for opt-in skip behavior).  One fat samples
list with many keys can serve many metrics — each reads its subset.</p>
<h2 id="fastvideo.eval.io.samples_from_manifest"><code>fastvideo.eval.io.samples_from_manifest</code></h2>
<pre class="doc-signature"><code>samples_from_manifest(manifest: str | Path, *, extract_audio: bool | str | Path = False, extract_workers: int = 4) -&gt; list&#91;dict&#93;</code></pre>
<p>Load canonical eval samples from a JSON or JSONL manifest.</p>
<p>Each row may contain <code>video</code>, <code>audio</code>, <code>reference</code>,
<code>reference_audio</code>, <code>reference_audio_source</code>, <code>text_prompt</code> and
arbitrary metric metadata. <code>reference_audio_source</code> is a media file
whose audio track is extracted when <code>extract_audio</code> is enabled. Paths are
resolved relative to the manifest directory. Video paths are wrapped as
:class:<code>Video</code> handles for lazy decoding; all other keys are preserved.
This is the recommended interface for benchmark generation, where matching
by sample id is safer than independently sorting folders.</p>
<h2 id="fastvideo.eval.io.build_eval_kwargs"><code>fastvideo.eval.io.build_eval_kwargs</code></h2>
<pre class="doc-signature"><code>build_eval_kwargs(row: dict, video_path: Path, *, fps: float = 24.0) -&gt; dict&#91;str, Any&#93;</code></pre>
<p>Build evaluator kwargs from a sample row + a video on disk.</p>
<p>Loads the video as <code>(T,C,H,W)</code> and adds the leading batch dim.
Forwards <code>prompt</code> (as scalar <code>text_prompt</code>) and
<code>auxiliary_info</code> (as scalar dict) when present on the row —
matches the one-sample-per-call contract that the evaluator and
every metric assume.</p>
<h2 id="fastvideo.eval.io.default_filename"><code>fastvideo.eval.io.default_filename</code></h2>
<pre class="doc-signature"><code>default_filename(row: dict, idx: int, ext: str = &#x27;.mp4&#x27;) -&gt; str</code></pre>
<p><code>&lt;sanitized-prompt&gt;-&lt;idx&gt;.mp4</code> — VBench-style.</p>
<h2 id="fastvideo.eval.io.glob_videos"><code>fastvideo.eval.io.glob_videos</code></h2>
<pre class="doc-signature"><code>glob_videos(videos_dir: Path, row: dict, ext: str = &#x27;.mp4&#x27;) -&gt; list&#91;Path&#93;</code></pre>
<p>Find every generated video for <em>row</em>, sorted by trailing <code>-&lt;idx&gt;</code>.</p>
<h2 id="fastvideo.eval.io.sanitize_prompt"><code>fastvideo.eval.io.sanitize_prompt</code></h2>
<pre class="doc-signature"><code>sanitize_prompt(prompt: str, max_len: int = 100) -&gt; str</code></pre>
<p>Prompt → safe filename stem.</p>
<h2 id="fastvideo.eval.io.extract_frames"><code>fastvideo.eval.io.extract_frames</code></h2>
<pre class="doc-signature"><code>extract_frames(video: torch.Tensor, n_frames: int | None = None) -&gt; torch.Tensor</code></pre>
<p>Uniformly sample <em>n_frames</em> from a <code>(T, C, H, W)</code> video tensor.</p>
<h2 id="fastvideo.eval.io.load_video"><code>fastvideo.eval.io.load_video</code></h2>
<pre class="doc-signature"><code>load_video(source: str | torch.Tensor | list, **kwargs) -&gt; torch.Tensor</code></pre>
<p>Load a video as a <code>(T, C, H, W)</code> float32 tensor in <code>[0, 1]</code>.</p>
<p>Supported <em>source</em> types:</p>
<ul>
<li><strong>str / Path</strong> – path to <code>.mp4</code> / <code>.avi</code> / <code>.gif</code> file, or a
  directory of frame images (sorted alphabetically).</li>
<li><strong>torch.Tensor</strong> – returned as-is after shape validation.</li>
<li><strong>list[PIL.Image]</strong> – stacked into a tensor.</li>
</ul>
</div>

<h2>Submodules</h2>
<ul>
<li><a href="/FastVideo/api/fastvideo/eval/io/audio">audio</a></li>
<li><a href="/FastVideo/api/fastvideo/eval/io/inputs">inputs</a></li>
<li><a href="/FastVideo/api/fastvideo/eval/io/paths">paths</a></li>
<li><a href="/FastVideo/api/fastvideo/eval/io/video">video</a></li>
</ul>