VBench Temporal Flickering — measures frame-to-frame stability.

Score = (255 - mean_MAE) / 255, where MAE is computed between consecutive frames in uint8 [0, 255] space. Higher = less flickering.

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.BaseMetric`

BaseMetric()

Abstract base class for all eval metrics.

Two execution shapes:

Per-sample (is_set_metric=False, default) — implement :meth:compute. The Evaluator calls it once per input sample and returns one :class:MetricResult per sample.
Set-vs-set (is_set_metric=True) — implement :meth:accumulate (called once per sample to buffer features) and :meth:finalize (called once after all samples to compute the corpus-level result). Use :meth:reset to clear buffers and :meth:merge_from to fold multi-GPU per-worker state together.

Optionally override :meth:setup to eagerly load models. Metrics that chunk along the time dim for memory hardcode their own chunk size in __init__ (see optical_flow for the canonical example). Eval always processes one video per :meth:Evaluator.evaluate call; compute / accumulate receive a single sample, not a batch.

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.BaseMetric.to`

to(device: str | torch.device) -> BaseMetric

Move metric (and its internal models) to device.

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.BaseMetric.setup`

setup() -> None

Eagerly load models. Called once by :class:EvalWorker.

Default is a no-op; metrics with no eager state (pixel math, closed-form ops) inherit this. Override only if your metric needs to load weights.

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.BaseMetric.compute`

compute(sample: dict) -> MetricResult

Per-sample metrics: compute the score for one sample.

sample["video"] is (T, C, H, W) float in [0, 1]. sample["reference"] (if used) has the same shape. Return self._skip(sample, reason) for missing inputs.

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.BaseMetric.reset`

reset() -> None

Clear accumulator state at the start of each evaluate() call.

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.BaseMetric.accumulate`

accumulate(sample: dict) -> None

Buffer per-sample features for a corpus-level metric.

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.BaseMetric.finalize`

finalize() -> MetricResult

Compute the corpus-level result from buffered state.

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.BaseMetric.merge_from`

merge_from(other: BaseMetric) -> None

Multi-GPU: fold another worker's accumulator state into this one.

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.register`

register(name: str)

Decorator to register a metric class.

Usage::

@register("ssim")
class SSIMMetric(BaseMetric):
    ...

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.MetricResult`

MetricResult(name: str, score: float | None, details: dict[str, Any] = dict())

Standard result container returned by all metrics.

score is None when the metric was skipped (e.g. missing required input). Check details["skipped"] for the reason.

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.TemporalFlickeringMetric`

TemporalFlickeringMetric()

Source

@register("vbench.temporal_flickering")
class TemporalFlickeringMetric(BaseMetric):

    name = "vbench.temporal_flickering"
    requires_reference = False
    higher_is_better = True
    needs_gpu = False

    @torch.no_grad()
    def compute(self, sample: dict) -> MetricResult:
        video = sample["video"]  # (T, C, H, W) [0, 1]
        T = video.shape[0]
        if T <= 1:
            return MetricResult(name=self.name, score=1.0, details={})

        frames = (video * 255.0).to(torch.uint8).cpu().numpy()
        frames = frames.transpose(0, 2, 3, 1).astype(np.float32)
        mae_per_pair = [float(np.mean(np.abs(frames[t] - frames[t + 1]))) for t in range(T - 1)]
        mean_mae = float(np.mean(mae_per_pair))
        return MetricResult(
            name=self.name,
            score=(255.0 - mean_mae) / 255.0,
            details={"per_pair_mae": mae_per_pair},
        )

`fastvideo.eval.metrics.vbench.temporal_flickering.metric.TemporalFlickeringMetric.compute`

compute(sample: dict) -> MetricResult

Source

@torch.no_grad()
def compute(self, sample: dict) -> MetricResult:
    video = sample["video"]  # (T, C, H, W) [0, 1]
    T = video.shape[0]
    if T <= 1:
        return MetricResult(name=self.name, score=1.0, details={})

    frames = (video * 255.0).to(torch.uint8).cpu().numpy()
    frames = frames.transpose(0, 2, 3, 1).astype(np.float32)
    mae_per_pair = [float(np.mean(np.abs(frames[t] - frames[t + 1]))) for t in range(T - 1)]
    mean_mae = float(np.mean(mae_per_pair))
    return MetricResult(
        name=self.name,
        score=(255.0 - mean_mae) / 255.0,
        details={"per_pair_mae": mae_per_pair},
    )