VBench Aesthetic Quality — CLIP ViT-L/14 + LAION aesthetic predictor.

Encodes frames through CLIP, passes L2-normalized features through a linear aesthetic head (768 → 1), and averages scores / 10.

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.BaseMetric`

BaseMetric()

Abstract base class for all eval metrics.

Two execution shapes:

Per-sample (is_set_metric=False, default) — implement :meth:compute. The Evaluator calls it once per input sample and returns one :class:MetricResult per sample.
Set-vs-set (is_set_metric=True) — implement :meth:accumulate (called once per sample to buffer features) and :meth:finalize (called once after all samples to compute the corpus-level result). Use :meth:reset to clear buffers and :meth:merge_from to fold multi-GPU per-worker state together.

Optionally override :meth:setup to eagerly load models. Metrics that chunk along the time dim for memory hardcode their own chunk size in __init__ (see optical_flow for the canonical example). Eval always processes one video per :meth:Evaluator.evaluate call; compute / accumulate receive a single sample, not a batch.

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.BaseMetric.to`

to(device: str | torch.device) -> BaseMetric

Move metric (and its internal models) to device.

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.BaseMetric.setup`

setup() -> None

Eagerly load models. Called once by :class:EvalWorker.

Default is a no-op; metrics with no eager state (pixel math, closed-form ops) inherit this. Override only if your metric needs to load weights.

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.BaseMetric.compute`

compute(sample: dict) -> MetricResult

Per-sample metrics: compute the score for one sample.

sample["video"] is (T, C, H, W) float in [0, 1]. sample["reference"] (if used) has the same shape. Return self._skip(sample, reason) for missing inputs.

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.BaseMetric.reset`

reset() -> None

Clear accumulator state at the start of each evaluate() call.

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.BaseMetric.accumulate`

accumulate(sample: dict) -> None

Buffer per-sample features for a corpus-level metric.

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.BaseMetric.finalize`

finalize() -> MetricResult

Compute the corpus-level result from buffered state.

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.BaseMetric.merge_from`

merge_from(other: BaseMetric) -> None

Multi-GPU: fold another worker's accumulator state into this one.

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.register`

register(name: str)

Decorator to register a metric class.

Usage::

@register("ssim")
class SSIMMetric(BaseMetric):
    ...

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.MetricResult`

MetricResult(name: str, score: float | None, details: dict[str, Any] = dict())

Standard result container returned by all metrics.

score is None when the metric was skipped (e.g. missing required input). Check details["skipped"] for the reason.

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.AestheticQualityMetric`

AestheticQualityMetric()

Source

@register("vbench.aesthetic_quality")
class AestheticQualityMetric(BaseMetric):

    name = "vbench.aesthetic_quality"
    requires_reference = False
    higher_is_better = True
    needs_gpu = True
    dependencies = ["clip"]
    backbone = "clip_vit_l14"

    def __init__(self) -> None:
        super().__init__()
        self._clip_model: Any = None
        self._aesthetic_head: Any = None

    def to(self, device):
        super().to(device)
        if self._clip_model is not None:
            self._clip_model = self._clip_model.to(self.device)
        if self._aesthetic_head is not None:
            self._aesthetic_head = self._aesthetic_head.to(self.device)
        return self

    def setup(self) -> None:
        if self._clip_model is not None:
            return

        import clip
        from fastvideo.eval.models import ensure_checkpoint, get_cache_dir
        self._clip_model, _ = clip.load(
            "ViT-L/14",
            device=self.device,
            download_root=str(get_cache_dir() / "clip"),
        )
        self._clip_model.eval()

        # Load LAION aesthetic head
        ckpt_path = ensure_checkpoint(
            "sa_0_4_vit_l_14_linear.pth",
            source=_AESTHETIC_URL,
        )
        self._aesthetic_head = nn.Linear(768, 1)
        self._aesthetic_head.load_state_dict(torch.load(ckpt_path, map_location="cpu", weights_only=True))
        self._aesthetic_head.to(self.device)
        self._aesthetic_head.eval()

    @torch.no_grad()
    def compute(self, sample: dict) -> MetricResult:
        video = sample["video"]  # (T, C, H, W)
        frames = _clip_transform(video.to(self.device))

        chunk = self._chunk_size or 32
        scores_list = []
        for i in range(0, frames.shape[0], chunk):
            feats = self._clip_model.encode_image(frames[i:i + chunk]).float()
            feats = F.normalize(feats, dim=-1, p=2)
            scores_list.append(self._aesthetic_head(feats).squeeze(-1))

        all_scores = torch.cat(scores_list, dim=0) / 10.0  # (T,)
        return MetricResult(
            name=self.name,
            score=float(all_scores.mean().item()),
            details={"per_frame": all_scores.tolist()},
        )

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.AestheticQualityMetric.to`

to(device)

Source

def to(self, device):
    super().to(device)
    if self._clip_model is not None:
        self._clip_model = self._clip_model.to(self.device)
    if self._aesthetic_head is not None:
        self._aesthetic_head = self._aesthetic_head.to(self.device)
    return self

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.AestheticQualityMetric.setup`

setup() -> None

Source

def setup(self) -> None:
    if self._clip_model is not None:
        return

    import clip
    from fastvideo.eval.models import ensure_checkpoint, get_cache_dir
    self._clip_model, _ = clip.load(
        "ViT-L/14",
        device=self.device,
        download_root=str(get_cache_dir() / "clip"),
    )
    self._clip_model.eval()

    # Load LAION aesthetic head
    ckpt_path = ensure_checkpoint(
        "sa_0_4_vit_l_14_linear.pth",
        source=_AESTHETIC_URL,
    )
    self._aesthetic_head = nn.Linear(768, 1)
    self._aesthetic_head.load_state_dict(torch.load(ckpt_path, map_location="cpu", weights_only=True))
    self._aesthetic_head.to(self.device)
    self._aesthetic_head.eval()

`fastvideo.eval.metrics.vbench.aesthetic_quality.metric.AestheticQualityMetric.compute`

compute(sample: dict) -> MetricResult

Source

@torch.no_grad()
def compute(self, sample: dict) -> MetricResult:
    video = sample["video"]  # (T, C, H, W)
    frames = _clip_transform(video.to(self.device))

    chunk = self._chunk_size or 32
    scores_list = []
    for i in range(0, frames.shape[0], chunk):
        feats = self._clip_model.encode_image(frames[i:i + chunk]).float()
        feats = F.normalize(feats, dim=-1, p=2)
        scores_list.append(self._aesthetic_head(feats).squeeze(-1))

    all_scores = torch.cat(scores_list, dim=0) / 10.0  # (T,)
    return MetricResult(
        name=self.name,
        score=float(all_scores.mean().item()),
        details={"per_frame": all_scores.tolist()},
    )