Skip to content

physics_iq

Physics-IQ benchmark prompt corpus.

Yields one sample dict per take-1 scenario, paired with its take-2 reference and both takes' real motion masks. Each row drops straight into :meth:fastvideo.eval.Evaluator.evaluate for the physics_iq metric:

{
    "prompt": <description>,
    "reference":            "<take-1 mp4>",
    "reference_take2":      "<take-2 mp4>",
    "reference_mask":       "<take-1 mask mp4>",
    "reference_take2_mask": "<take-2 mask mp4>",
    "scenario": <scenario_id>,
    "view": <camera view>,
    "auxiliary_info": { ... metadata ... },
}

Self-contained dataset: the manifest CSV is vendored under fastvideo/eval/metrics/physics_iq/_vendored/descriptions.csv; per-scenario videos/masks/switch-frames auto-fetch on first use from the public DeepMind bucket into ${FASTVIDEO_EVAL_CACHE}/datasets/physics_iq/. Pass auto_download=False (or dataset_root= pointing at a pre-downloaded copy) to opt out of network fetches.

Classes

fastvideo.eval.datasets.physics_iq.PhysicsIQPromptDataset 

PhysicsIQPromptDataset(dataset_root: str | Path | None = None, *, fps: int = _DEFAULT_FPS, limit: int | None = None, generated_dir: str | Path | None = None, auto_download: bool = True)

Bases: PromptDataset

Physics-IQ benchmark prompt corpus.

Self-contained: get_dataset("physics_iq") works with no kwargs. The manifest CSV is vendored next to the metric, and per-scenario assets auto-fetch on first miss from the public bucket into ${FASTVIDEO_EVAL_CACHE}/datasets/physics_iq/.

Parameters:

Name Type Description Default
dataset_root str | Path | None

path to a pre-downloaded copy of the Physics-IQ release. Defaults to ${FASTVIDEO_EVAL_CACHE}/datasets/physics_iq; override only if you already have a local mirror.

 None
fps int

target frame rate. The release ships at 30 FPS; other rates transcode once on first access into <root>/.physics_iq_cache/.

 _DEFAULT_FPS
limit int | None

optional truncation for quick smoke runs. Apply this kwarg (not a post-construction slice) so we only fetch the assets for the scenarios actually requested.

 None
generated_dir str | Path | None

optional directory of pre-generated videos — attaches each manifest row's expected output path to the sample dict under auxiliary_info["generated_video_path"].

 None
auto_download bool

when True (the default), missing testing videos, masks, and switch frames are fetched from the public bucket into dataset_root. Set False for air-gapped runs; the loader will then raise FileNotFoundError on miss.

 True
Source code in fastvideo/eval/datasets/physics_iq.py 
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
def __init__(
    self,
    dataset_root: str | Path | None = None,
    *,
    fps: int = _DEFAULT_FPS,
    limit: int | None = None,
    generated_dir: str | Path | None = None,
    auto_download: bool = True,
) -> None:
    super().__init__()

    repo_root = Path(dataset_root or _default_dataset_root()).expanduser().resolve()
    self.repo_root = repo_root
    self.dataset_dir = _resolve_dataset_dir(repo_root)
    self.descriptions_path = _resolve_descriptions_path(repo_root, self.dataset_dir)
    self.cache_dir = repo_root / ".physics_iq_cache"
    self.fps = fps
    self.auto_download = auto_download
    self.bucket_url = _bucket_url()

    scenarios = self._iter_scenarios(
        fps=fps,
        generated_dir=generated_dir,
        limit=limit,
    )
    self._rows = [_scenario_to_row(s) for s in scenarios]

fastvideo.eval.datasets.physics_iq.PhysicsIQScenario dataclass 

PhysicsIQScenario(scenario_id: str, view: str, scenario_name: str, take1_video_path: str, take2_video_path: str, switch_frame_path: str, caption: str, expected_gen_filename: str, generated_video_path: str | None = None, take1_mask_path: str | None = None, take2_mask_path: str | None = None)

One row of the Physics-IQ manifest, fully resolved on disk.

Functions: