--- title: router ---

Multi-replica load balancer + WebSocket proxy for the streaming server.

Sits in front of one-or-more streaming-server replicas and forwards WebSocket sessions to a healthy primary, with failover to secondaries. Kept in-repo under fastvideo/entrypoints/streaming/router/ per the PR plan's default; the alternative (separate package) is an open question deferred to review.

fastvideo.entrypoints.streaming.router.Replica

Replica(endpoint: ReplicaEndpoint, health: ReplicaHealth = ReplicaHealth())

fastvideo.entrypoints.streaming.router.ReplicaHealth

ReplicaHealth(status: ReplicaStatus = ReplicaStatus.UNKNOWN, last_ok_at: float | None = None, last_failure_at: float | None = None, consecutive_failures: int = 0, consecutive_successes: int = 0, last_latency_ms: float | None = None)

fastvideo.entrypoints.streaming.router.ReplicaRegistry

ReplicaRegistry(replicas: list[ReplicaEndpoint])

Stateful map of replica URL → :class:Replica.

Selection favors primary replicas when healthy; otherwise the first healthy non-primary is returned. When none are healthy, the registry returns None so the router can reject incoming sessions with gpu_unavailable.

fastvideo.entrypoints.streaming.router.ReplicaRegistry.all

all() -> list[Replica]

fastvideo.entrypoints.streaming.router.ReplicaRegistry.get

get(url: str) -> Replica | None

fastvideo.entrypoints.streaming.router.ReplicaRegistry.primaries

primaries() -> list[Replica]

fastvideo.entrypoints.streaming.router.ReplicaRegistry.select

select() -> Replica | None

Pick the best healthy replica.

Priority order:

  1. The first healthy primary (insertion order).
  2. The first healthy non-primary (insertion order).
  3. None when nothing is healthy.

This MVP picks the first match within each tier; it does NOT load-balance across multiple healthy replicas of the same tier. Round-robin and weighted distribution are deferred until a real N-way active deployment exists.

fastvideo.entrypoints.streaming.router.ReplicaRegistry.record_success

record_success(replica: Replica, *, recovery_threshold: int, latency_ms: float) -> None

fastvideo.entrypoints.streaming.router.ReplicaRegistry.record_failure

record_failure(replica: Replica, *, failure_threshold: int, reason: str) -> None

fastvideo.entrypoints.streaming.router.ReplicaStatus

ReplicaStatus()

fastvideo.entrypoints.streaming.router.RouterConfig

RouterConfig(host: str = '0.0.0.0', port: int = 9000, replicas: list[ReplicaEndpoint] = list(), health_check_path: str = '/health', health_check_interval_seconds: float = 5.0, health_check_timeout_seconds: float = 2.0, failure_threshold: int = 3, recovery_threshold: int = 2)

Typed router config loaded from a YAML file.

Example::

router:
  host: 0.0.0.0
  port: 9000
  replicas:
    - url: http://streamer-a:8000
      primary: true
    - url: http://streamer-b:8000
  health_check:
    path: /health
    interval_seconds: 5
    failure_threshold: 3

Validation runs in __post_init__: empty replicas, non-positive intervals/timeouts, thresholds < 1, non-http(s) URLs, and more than one primary all raise ValueError so misconfigurations surface at load time rather than as confusing runtime failures.

fastvideo.entrypoints.streaming.router.build_router_app

build_router_app(config: RouterConfig, *, registry: ReplicaRegistry | None = None) -> FastAPI

Build the router FastAPI app.

registry can be injected for tests; defaults to one built from config.replicas.

fastvideo.entrypoints.streaming.router.run_router

run_router(config: RouterConfig) -> None

Submodules