rewrite

Rewrite payload builder.

The UI's "rewrite seed prompts" flow asks the enhancer to produce a batch of alternative prompts given one seed. This module packages the seed + options into the payload the enhancer expects and unpacks the response back into a typed :class:RewriteResult.

Separating this from :mod:enhancer keeps the enhancer provider- agnostic; anything UI-specific (how many alternatives to request, how to split the response, temperature) lives here.

fastvideo.entrypoints.streaming.prompt.rewrite.PromptEnhancer

PromptEnhancer(*, providers: Sequence[LLMProvider], model: str, timeout_ms: int = 20000, temperature: float = 0.7, max_tokens: int | None = 256, system_prompt_dir: str | None = None)

Orchestrates prompt operations across a priority-ordered provider list with structured fallback + hot-reloadable system prompts.

Usage::

enhancer = PromptEnhancer(
    providers=[CerebrasProvider(), GroqProvider()],
    model="gpt-oss-120b",
    system_prompt_dir="/etc/fastvideo/prompts",
)
response = await enhancer.enhance("a fox running through snow")

fastvideo.entrypoints.streaming.prompt.rewrite.PromptEnhancer.register_provider

register_provider(provider: LLMProvider, *, priority: int = -1) -> None

Insert an additional provider. priority=0 makes it primary; priority=-1 (default) appends as a fallback.

fastvideo.entrypoints.streaming.prompt.rewrite.PromptEnhancer.reload_system_prompts

reload_system_prompts() -> None

Re-read the system prompt files from system_prompt_dir.

The streaming server exposes this via a management endpoint so operators can iterate on prompt templates without restarting workers.

fastvideo.entrypoints.streaming.prompt.rewrite.PromptEnhancer.enhance

enhance(prompt: str) -> LLMResponse

fastvideo.entrypoints.streaming.prompt.rewrite.PromptEnhancer.auto_extend

auto_extend(prior_prompts: Sequence[str]) -> LLMResponse

fastvideo.entrypoints.streaming.prompt.rewrite.PromptEnhancer.rewrite

rewrite(seed_prompt: str) -> LLMResponse

fastvideo.entrypoints.streaming.prompt.rewrite.RewriteOptions

RewriteOptions(count: int = 3, temperature: float | None = None)

@dataclass
class RewriteOptions:
    count: int = 3
    """Number of alternative prompts to request."""
    temperature: float | None = None

fastvideo.entrypoints.streaming.prompt.rewrite.RewriteOptions.count

count: int = 3

Number of alternative prompts to request.

count: int = 3

fastvideo.entrypoints.streaming.prompt.rewrite.RewriteResult

RewriteResult(seed_prompt: str, alternatives: list[str], provider: str, model: str, latency_ms: float, fallback_used: bool = False)

@dataclass
class RewriteResult:
    seed_prompt: str
    alternatives: list[str]
    provider: str
    model: str
    latency_ms: float
    fallback_used: bool = False

fastvideo.entrypoints.streaming.prompt.rewrite.build_rewrite

build_rewrite(enhancer: PromptEnhancer, seed_prompt: str, *, options: RewriteOptions | None = None) -> RewriteResult

Run a rewrite op through the enhancer and return a typed result.

async def build_rewrite(
    enhancer: PromptEnhancer,
    seed_prompt: str,
    *,
    options: RewriteOptions | None = None,
) -> RewriteResult:
    """Run a rewrite op through the enhancer and return a typed result."""
    if not seed_prompt.strip():
        raise ValueError("rewrite seed prompt must be non-empty")
    options = options or RewriteOptions()
    response = await enhancer.rewrite(seed_prompt)
    alternatives = _split_response(response.content, limit=options.count)
    return RewriteResult(
        seed_prompt=seed_prompt,
        alternatives=alternatives,
        provider=response.provider,
        model=response.model,
        latency_ms=response.latency_ms,
        fallback_used=response.fallback_used,
    )