base

Prompt-corpus datasets.

A :class:PromptDataset is an iterable of sample dicts describing the prompts and conditions for a benchmark. Each sample is a plain dict — no dataclass, no schema enforcement — that flows directly into both generation (VideoGenerator.generate_video(**sample)) and scoring (Evaluator.evaluate(**eval_kwargs)). The runner picks well-known keys (prompt, n_samples, dimensions, auxiliary_info, ...) and passes the rest through.

This matches the surrounding FastVideo style:

:class:fastvideo.dataset.validation_dataset.ValidationDataset yields dicts.
:meth:fastvideo.VideoGenerator.generate_video consumes **kwargs.
:meth:fastvideo.eval.Evaluator.evaluate consumes **kwargs.

To add a new benchmark:

Subclass :class:PromptDataset, populate self._rows with dicts in __init__.
Decorate with @register_dataset("my_bench").

Convention for auxiliary_info: a flat dict of metric-keyed values (e.g. {"color": "red"}). Benchmarks with nested aux schemas (VBench's {dim: {key: val}}) flatten at load time so every consumer sees the same shape.

`fastvideo.eval.datasets.base.Sample`

Sample()

Documented schema for a row yielded by :class:PromptDataset.

Only prompt is required. Extra keys beyond these are forwarded to the runner's eval-kwargs builder verbatim, so action-conditioned or audio-bearing benchmarks can add their own fields without changing the base class.

Source

class Sample(TypedDict, total=False):
    """Documented schema for a row yielded by :class:`PromptDataset`.
Only ``prompt`` is required. Extra keys beyond these are forwarded to
the runner&#x27;s eval-kwargs builder verbatim, so action-conditioned or
audio-bearing benchmarks can add their own fields without changing
the base class.
&quot;&quot;&quot;
prompt: str
n_samples: int
dimensions: list&#91;str&#93;
auxiliary_info: dict
image_path: str
reference_video: str</code></pre></details>
fastvideo.eval.datasets.base.PromptDataset
PromptDataset()
Iterable corpus of sample dicts. Subclasses populate self._rows.
Source
class PromptDataset:
    """Iterable corpus of sample dicts. Subclasses populate ``self._rows``."""
name: str = &quot;&quot;
description: str = &quot;&quot;
supports_dimensions: bool = False
requires_reference_image: bool = False
requires_reference_video: bool = False

def __init__(self) -&gt; None:
    self._rows: list&#91;dict&#93; = &#91;&#93;

def __iter__(self) -&gt; Iterator&#91;dict&#93;:
    return iter(self._rows)

def __len__(self) -&gt; int:
    return len(self._rows)

def __getitem__(self, i: int) -&gt; dict:
    return self._rows&#91;i&#93;

def by_dimension(self) -&gt; dict&#91;str, list&#91;dict&#93;&#93;:
    &quot;&quot;&quot;Group samples by dimension. A multi-dim sample appears under each.&quot;&quot;&quot;
    out: dict&#91;str, list&#91;dict&#93;&#93; = &#123;&#125;
    for s in self._rows:
        for d in s.get(&quot;dimensions&quot;, ()):
            out.setdefault(d, &#91;&#93;).append(s)
    return out</code></pre></details>
fastvideo.eval.datasets.base.PromptDataset.by_dimension
by_dimension() -> dict[str, list[dict]]
Group samples by dimension. A multi-dim sample appears under each.
Source
def by_dimension(self) -> dict[str, list[dict]]:
    """Group samples by dimension. A multi-dim sample appears under each."""
    out: dict[str, list[dict]] = {}
    for s in self._rows:
        for d in s.get("dimensions", ()):
            out.setdefault(d, []).append(s)
    return out

base

`fastvideo.eval.datasets.base.Sample`

`fastvideo.eval.datasets.base.PromptDataset`

`fastvideo.eval.datasets.base.PromptDataset.by_dimension`