--- title: metric ---

VBench Subject Consistency — DINO ViT-B/16 temporal feature similarity.

Measures how well the main subject maintains its appearance throughout the video via cosine similarity of DINO features between consecutive frames and the first frame.

fastvideo.eval.metrics.vbench.subject_consistency.metric.BaseMetric

BaseMetric()

Abstract base class for all eval metrics.

Two execution shapes:

Optionally override :meth:setup to eagerly load models. Metrics that chunk along the time dim for memory hardcode their own chunk size in __init__ (see optical_flow for the canonical example). Eval always processes one video per :meth:Evaluator.evaluate call; compute / accumulate receive a single sample, not a batch.

fastvideo.eval.metrics.vbench.subject_consistency.metric.BaseMetric.to

to(device: str | torch.device) -> BaseMetric

Move metric (and its internal models) to device.

fastvideo.eval.metrics.vbench.subject_consistency.metric.BaseMetric.setup

setup() -> None

Eagerly load models. Called once by :class:EvalWorker.

Default is a no-op; metrics with no eager state (pixel math, closed-form ops) inherit this. Override only if your metric needs to load weights.

fastvideo.eval.metrics.vbench.subject_consistency.metric.BaseMetric.compute

compute(sample: dict) -> MetricResult

Per-sample metrics: compute the score for one sample.

sample["video"] is (T, C, H, W) float in [0, 1]. sample["reference"] (if used) has the same shape. Return self._skip(sample, reason) for missing inputs.

fastvideo.eval.metrics.vbench.subject_consistency.metric.BaseMetric.reset

reset() -> None

Clear accumulator state at the start of each evaluate() call.

fastvideo.eval.metrics.vbench.subject_consistency.metric.BaseMetric.accumulate

accumulate(sample: dict) -> None

Buffer per-sample features for a corpus-level metric.

fastvideo.eval.metrics.vbench.subject_consistency.metric.BaseMetric.finalize

finalize() -> MetricResult

Compute the corpus-level result from buffered state.

fastvideo.eval.metrics.vbench.subject_consistency.metric.BaseMetric.merge_from

merge_from(other: BaseMetric) -> None

Multi-GPU: fold another worker's accumulator state into this one.

fastvideo.eval.metrics.vbench.subject_consistency.metric.register

register(name: str)

Decorator to register a metric class.

Usage::

@register("ssim")
class SSIMMetric(BaseMetric):
    ...

fastvideo.eval.metrics.vbench.subject_consistency.metric.MetricResult

MetricResult(name: str, score: float | None, details: dict[str, Any] = dict())

Standard result container returned by all metrics.

score is None when the metric was skipped (e.g. missing required input). Check details["skipped"] for the reason.

fastvideo.eval.metrics.vbench.subject_consistency.metric.consistency_score

consistency_score(features: torch.Tensor) -> float

VBench-style temporal consistency from (T, D) L2-normalized features.

For each frame t > 0, computes: sim = (cos(f[t], f[t-1]) + cos(f[t], f[0])) / 2, clamped >= 0

Returns the mean similarity across all t > 0.

fastvideo.eval.metrics.vbench.subject_consistency.metric.SubjectConsistencyMetric

SubjectConsistencyMetric()
Source
@register("vbench.subject_consistency")
class SubjectConsistencyMetric(BaseMetric):

    name = "vbench.subject_consistency"
    requires_reference = False
    higher_is_better = True
    needs_gpu = True
    backbone = "dino_vitb16"

    def __init__(self) -> None:
        super().__init__()
        self._model: Any = None

    def to(self, device):
        super().to(device)
        if self._model is not None:
            self._model = self._model.to(self.device)
        return self

    def setup(self) -> None:
        if self._model is not None:
            return
        model = torch.hub.load("facebookresearch/dino:main", "dino_vitb16")
        model.to(self.device)
        model.eval()
        self._model = model

    @torch.no_grad()
    def compute(self, sample: dict) -> MetricResult:
        video = sample["video"]  # (T, C, H, W)
        frames = video.to(self.device)
        # antialias=False matches VBench's dino_transform (vbench/utils.py:50)
        frames = resize(frames, 224, antialias=False)
        frames = normalize(frames, mean=_MEAN, std=_STD)

        chunk = self._chunk_size or 64
        feats = []
        for i in range(0, frames.shape[0], chunk):
            f = self._model(frames[i:i + chunk])
            f = F.normalize(f, dim=-1, p=2)
            feats.append(f)
        all_feats = torch.cat(feats, dim=0)  # (T, D)
        return MetricResult(
            name=self.name,
            score=consistency_score(all_feats),
            details={},
        )

fastvideo.eval.metrics.vbench.subject_consistency.metric.SubjectConsistencyMetric.to

to(device)
Source
def to(self, device):
    super().to(device)
    if self._model is not None:
        self._model = self._model.to(self.device)
    return self

fastvideo.eval.metrics.vbench.subject_consistency.metric.SubjectConsistencyMetric.setup

setup() -> None
Source
def setup(self) -> None:
    if self._model is not None:
        return
    model = torch.hub.load("facebookresearch/dino:main", "dino_vitb16")
    model.to(self.device)
    model.eval()
    self._model = model

fastvideo.eval.metrics.vbench.subject_consistency.metric.SubjectConsistencyMetric.compute

compute(sample: dict) -> MetricResult
Source
@torch.no_grad()
def compute(self, sample: dict) -> MetricResult:
    video = sample["video"]  # (T, C, H, W)
    frames = video.to(self.device)
    # antialias=False matches VBench's dino_transform (vbench/utils.py:50)
    frames = resize(frames, 224, antialias=False)
    frames = normalize(frames, mean=_MEAN, std=_STD)

    chunk = self._chunk_size or 64
    feats = []
    for i in range(0, frames.shape[0], chunk):
        f = self._model(frames[i:i + chunk])
        f = F.normalize(f, dim=-1, p=2)
        feats.append(f)
    all_feats = torch.cat(feats, dim=0)  # (T, D)
    return MetricResult(
        name=self.name,
        score=consistency_score(all_feats),
        details={},
    )