--- title: groq ---

Groq LLM provider (OpenAI-compatible chat endpoint).

fastvideo.entrypoints.streaming.prompt.providers.groq.complete_openai_compatible

complete_openai_compatible(*, api_key: str | None, api_key_hint: str, base_url: str, provider_name: str, request: LLMRequest) -> LLMResponse

Issue a chat-completions call and decode the OpenAI response.

fastvideo.entrypoints.streaming.prompt.providers.groq.LLMRequest

LLMRequest(messages: list[LLMMessage], model: str, max_tokens: int | None = None, temperature: float | None = None, timeout_ms: int | None = None)

fastvideo.entrypoints.streaming.prompt.providers.groq.LLMResponse

LLMResponse(content: str, provider: str, model: str, latency_ms: float, fallback_used: bool = False)

fastvideo.entrypoints.streaming.prompt.providers.groq.GroqProvider

GroqProvider(api_key: str | None = None, base_url: str = _DEFAULT_BASE_URL, name: str = 'groq')

Groq inference adapter.

Identical wire format to :class:CerebrasProvider; both go through :func:complete_openai_compatible. The two providers differ only in base URL, env var, and model id conventions.

Source
@dataclass
class GroqProvider:
    """Groq inference adapter.

    Identical wire format to :class:`CerebrasProvider`; both go through
    :func:`complete_openai_compatible`. The two providers differ only
    in base URL, env var, and model id conventions.
    """

    api_key: str | None = None
    base_url: str = _DEFAULT_BASE_URL
    name: str = "groq"

    def __post_init__(self) -> None:
        if self.api_key is None:
            self.api_key = os.environ.get(_API_KEY_ENV)

    async def complete(self, request: LLMRequest) -> LLMResponse:
        return await complete_openai_compatible(
            api_key=self.api_key,
            api_key_hint=_API_KEY_ENV,
            base_url=self.base_url,
            provider_name=self.name,
            request=request,
        )

fastvideo.entrypoints.streaming.prompt.providers.groq.GroqProvider.complete

complete(request: LLMRequest) -> LLMResponse
Source
async def complete(self, request: LLMRequest) -> LLMResponse:
    return await complete_openai_compatible(
        api_key=self.api_key,
        api_key_hint=_API_KEY_ENV,
        base_url=self.base_url,
        provider_name=self.name,
        request=request,
    )