Skip to content

base

Classes

fastvideo.eval.metrics.base.BaseMetric 

BaseMetric()

Abstract base class for all eval metrics.

Two execution shapes:

  • Per-sample (is_set_metric=False, default) — implement :meth:compute. The Evaluator calls it once per input sample and returns one :class:MetricResult per sample.
  • Set-vs-set (is_set_metric=True) — implement :meth:accumulate (called once per sample to buffer features) and :meth:finalize (called once after all samples to compute the corpus-level result). Use :meth:reset to clear buffers and :meth:merge_from to fold multi-GPU per-worker state together.

Optionally override :meth:setup to eagerly load models. Metrics that chunk along the time dim for memory hardcode their own chunk size in __init__ (see optical_flow for the canonical example). Eval always processes one video per :meth:Evaluator.evaluate call; compute / accumulate receive a single sample, not a batch.

Source code in fastvideo/eval/metrics/base.py 
44
45
def __init__(self) -> None:
    self._device: torch.device = torch.device("cpu")

Methods:

fastvideo.eval.metrics.base.BaseMetric.accumulate 
accumulate(sample: dict) -> None

Buffer per-sample features for a corpus-level metric.

Source code in fastvideo/eval/metrics/base.py 
82
83
84
def accumulate(self, sample: dict) -> None:
    """Buffer per-sample features for a corpus-level metric."""
    raise NotImplementedError(f"{type(self).__name__}.accumulate is not implemented")
fastvideo.eval.metrics.base.BaseMetric.compute 
compute(sample: dict) -> MetricResult

Per-sample metrics: compute the score for one sample.

sample["video"] is (T, C, H, W) float in [0, 1]. sample["reference"] (if used) has the same shape. Return self._skip(sample, reason) for missing inputs.

Source code in fastvideo/eval/metrics/base.py 
68
69
70
71
72
73
74
75
def compute(self, sample: dict) -> MetricResult:
    """Per-sample metrics: compute the score for one sample.

    ``sample["video"]`` is ``(T, C, H, W)`` float in ``[0, 1]``.
    ``sample["reference"]`` (if used) has the same shape. Return
    ``self._skip(sample, reason)`` for missing inputs.
    """
    raise NotImplementedError(f"{type(self).__name__}.compute is not implemented")
fastvideo.eval.metrics.base.BaseMetric.finalize 
finalize() -> MetricResult

Compute the corpus-level result from buffered state.

Source code in fastvideo/eval/metrics/base.py 
86
87
88
def finalize(self) -> MetricResult:
    """Compute the corpus-level result from buffered state."""
    raise NotImplementedError(f"{type(self).__name__}.finalize is not implemented")
fastvideo.eval.metrics.base.BaseMetric.merge_from 
merge_from(other: BaseMetric) -> None

Multi-GPU: fold another worker's accumulator state into this one.

Source code in fastvideo/eval/metrics/base.py 
90
91
def merge_from(self, other: BaseMetric) -> None:  # noqa: B027 - intentionally optional override
    """Multi-GPU: fold another worker's accumulator state into this one."""
fastvideo.eval.metrics.base.BaseMetric.reset 
reset() -> None

Clear accumulator state at the start of each evaluate() call.

Source code in fastvideo/eval/metrics/base.py 
79
80
def reset(self) -> None:  # noqa: B027 - intentionally optional override
    """Clear accumulator state at the start of each evaluate() call."""
fastvideo.eval.metrics.base.BaseMetric.setup 
setup() -> None

Eagerly load models. Called once by :class:EvalWorker.

Default is a no-op; metrics with no eager state (pixel math, closed-form ops) inherit this. Override only if your metric needs to load weights.

Source code in fastvideo/eval/metrics/base.py 
56
57
58
59
60
61
62
def setup(self) -> None:  # noqa: B027 - intentionally optional override
    """Eagerly load models. Called once by :class:`EvalWorker`.

    Default is a no-op; metrics with no eager state (pixel math,
    closed-form ops) inherit this. Override only if your metric
    needs to load weights.
    """
fastvideo.eval.metrics.base.BaseMetric.to 
to(device: str | device) -> BaseMetric

Move metric (and its internal models) to device.

Source code in fastvideo/eval/metrics/base.py 
51
52
53
54
def to(self, device: str | torch.device) -> BaseMetric:
    """Move metric (and its internal models) to *device*."""
    self._device = torch.device(device)
    return self