metric

Pairwise VLM judge (Gemini): of two rollouts under the same control, which better separates the third-person character (foreground) from the background; the score is the candidate's win-rate over a reference.

Classes

fastvideo.eval.metrics.judge.third_person_separation.metric.ThirdPersonSeparationMetric

ThirdPersonSeparationMetric(model: str = DEFAULT_MODEL, k: int = DEFAULT_K)

Bases: BaseMetric

Pairwise VLM judge of third-person fg/bg separation; corpus win-rate.

Source code in fastvideo/eval/metrics/judge/third_person_separation/metric.py

def __init__(self, model: str = DEFAULT_MODEL, k: int = DEFAULT_K) -> None:
    super().__init__()
    self.model = model
    self.k = k
    self._client: Any = None
    self._files: dict[str, Any] = {}  # path -> uploaded Gemini file handle
    self._records: list[dict] = []  # one per accumulated pair

Functions: