--- title: providers ---

LLM provider implementations used by the prompt enhancer.

fastvideo.entrypoints.streaming.prompt.providers.LLMMessage

LLMMessage(role: Literal['system', 'user', 'assistant'], content: str)

fastvideo.entrypoints.streaming.prompt.providers.LLMProvider

LLMProvider()

Provider interface every LLM adapter implements.

Providers are async-first because every built-in implementation talks to an HTTP API. Synchronous providers can wrap their call in asyncio.to_thread internally.

fastvideo.entrypoints.streaming.prompt.providers.LLMProvider.complete

complete(request: LLMRequest) -> LLMResponse

fastvideo.entrypoints.streaming.prompt.providers.LLMProviderError

LLMProviderError(message: str, *, retryable: bool = True)

Raised when an LLM provider fails a request.

retryable controls whether the enhancer falls back to the next provider. It is settable per-instance so the same exception type can describe retryable transport errors (5xx, 429) and non-retryable client errors (4xx auth/bad-request) without forcing a separate subclass for every status family.

fastvideo.entrypoints.streaming.prompt.providers.LLMRequest

LLMRequest(messages: list[LLMMessage], model: str, max_tokens: int | None = None, temperature: float | None = None, timeout_ms: int | None = None)

fastvideo.entrypoints.streaming.prompt.providers.LLMResponse

LLMResponse(content: str, provider: str, model: str, latency_ms: float, fallback_used: bool = False)

fastvideo.entrypoints.streaming.prompt.providers.LLMTimeoutError

LLMTimeoutError(message: str)

Raised when an LLM provider times out — always retryable.

fastvideo.entrypoints.streaming.prompt.providers.CerebrasProvider

CerebrasProvider(api_key: str | None = None, base_url: str = _DEFAULT_BASE_URL, name: str = 'cerebras')

Cerebras inference adapter.

api_key falls back to CEREBRAS_API_KEY when unset.

fastvideo.entrypoints.streaming.prompt.providers.CerebrasProvider.complete

complete(request: LLMRequest) -> LLMResponse

fastvideo.entrypoints.streaming.prompt.providers.GroqProvider

GroqProvider(api_key: str | None = None, base_url: str = _DEFAULT_BASE_URL, name: str = 'groq')

Groq inference adapter.

Identical wire format to :class:CerebrasProvider; both go through :func:complete_openai_compatible. The two providers differ only in base URL, env var, and model id conventions.

fastvideo.entrypoints.streaming.prompt.providers.GroqProvider.complete

complete(request: LLMRequest) -> LLMResponse

Submodules