`fastvideo.eval.registry.BaseMetric`

BaseMetric()

Abstract base class for all eval metrics.

Two execution shapes:

Per-sample (is_set_metric=False, default) — implement :meth:compute. The Evaluator calls it once per input sample and returns one :class:MetricResult per sample.
Set-vs-set (is_set_metric=True) — implement :meth:accumulate (called once per sample to buffer features) and :meth:finalize (called once after all samples to compute the corpus-level result). Use :meth:reset to clear buffers and :meth:merge_from to fold multi-GPU per-worker state together.

Optionally override :meth:setup to eagerly load models. Metrics that chunk along the time dim for memory hardcode their own chunk size in __init__ (see optical_flow for the canonical example). Eval always processes one video per :meth:Evaluator.evaluate call; compute / accumulate receive a single sample, not a batch.

`fastvideo.eval.registry.BaseMetric.to`

to(device: str | torch.device) -> BaseMetric

Move metric (and its internal models) to device.

`fastvideo.eval.registry.BaseMetric.setup`

setup() -> None

Eagerly load models. Called once by :class:EvalWorker.

Default is a no-op; metrics with no eager state (pixel math, closed-form ops) inherit this. Override only if your metric needs to load weights.

`fastvideo.eval.registry.BaseMetric.compute`

compute(sample: dict) -> MetricResult

Per-sample metrics: compute the score for one sample.

sample["video"] is (T, C, H, W) float in [0, 1]. sample["reference"] (if used) has the same shape. Return self._skip(sample, reason) for missing inputs.

`fastvideo.eval.registry.BaseMetric.reset`

reset() -> None

Clear accumulator state at the start of each evaluate() call.

`fastvideo.eval.registry.BaseMetric.accumulate`

accumulate(sample: dict) -> None

Buffer per-sample features for a corpus-level metric.

`fastvideo.eval.registry.BaseMetric.finalize`

finalize() -> MetricResult

Compute the corpus-level result from buffered state.

`fastvideo.eval.registry.BaseMetric.merge_from`

merge_from(other: BaseMetric) -> None

Multi-GPU: fold another worker's accumulator state into this one.

`fastvideo.eval.registry.register`

register(name: str)

Decorator to register a metric class.

Usage::

@register("ssim")
class SSIMMetric(BaseMetric):
    ...

Source

def register(name: str):
    """Decorator to register a metric class.

    Usage::

        @register("ssim")
        class SSIMMetric(BaseMetric):
            ...
    """

    def wrapper(cls):
        _REGISTRY[name] = cls
        return cls

    return wrapper

`fastvideo.eval.registry.get_metric`

get_metric(name: str, **kwargs: Any) -> BaseMetric

Instantiate a registered metric by name.

Checks that optional dependencies are installed before instantiation and gives a clear install hint pointing at the right extra group.

Source

def get_metric(name: str, **kwargs: Any) -> BaseMetric:
    """Instantiate a registered metric by name.

    Checks that optional dependencies are installed before instantiation
    and gives a clear install hint pointing at the right extra group.
    """
    cls = _REGISTRY.get(name)
    if cls is None:
        available = ", ".join(sorted(_REGISTRY.keys()))
        raise KeyError(f"Unknown metric '{name}'. Available: {available}")

    for dep in getattr(cls, "dependencies", []):
        if not importlib.util.find_spec(dep):
            raise ImportError(f"{cls.__name__} requires '{dep}'. "
                              f"Install with: {_install_hint(name, dep)}")

    return cls(**kwargs)

`fastvideo.eval.registry.missing_dependencies`

missing_dependencies(metric_name: str) -> list[str]

Importable module names declared by metric_name that are not actually importable in this environment. Returns [] if all deps are satisfied or the metric is unknown.

Used by group-style resolution to decide which metrics to silently skip (vs. naming a metric explicitly, where the missing dep should surface as :class:ImportError).

Source

def missing_dependencies(metric_name: str) -> list[str]:
    """Importable module names declared by *metric_name* that are not
    actually importable in this environment. Returns ``[]`` if all deps
    are satisfied or the metric is unknown.

    Used by group-style resolution to decide which metrics to silently
    skip (vs. naming a metric explicitly, where the missing dep should
    surface as :class:`ImportError`).
    """
    cls = _REGISTRY.get(metric_name)
    if cls is None:
        return []
    return [d for d in getattr(cls, "dependencies", []) if not importlib.util.find_spec(d)]

`fastvideo.eval.registry.list_metrics`

list_metrics() -> list[str]

Return sorted list of all registered metric names.

Source

def list_metrics() -> list[str]:
    """Return sorted list of all registered metric names."""
    return sorted(_REGISTRY.keys())

`fastvideo.eval.registry.resolve_group`

resolve_group(name: str) -> list[str] | None

If name is a group prefix (e.g. "vbench"), return all matching metric names. Returns None if name is not a group.

Source

def resolve_group(name: str) -> list[str] | None:
    """If *name* is a group prefix (e.g. ``"vbench"``), return all matching
    metric names.  Returns ``None`` if *name* is not a group."""
    prefix = name + "."
    matches = sorted(k for k in _REGISTRY if k.startswith(prefix))
    return matches if matches else None