Prompt caching

Router supports provider prompt caching today. The selected model provider owns the cache; Router preserves supported cache controls and accounts for the cache read and write tokens reported by the provider.

Anthropic Messages

Requests sent to POST /v1/messages keep explicit Anthropic cache_control values unchanged when they resolve to an Anthropic model. Router does not add cache markers that the caller did not send.

When a Messages request resolves to another provider, Anthropic cache markers are advisory because they are not part of that provider's API. The selected provider's native prompt-caching behavior applies instead.

Responses API

Requests sent to POST /v1/responses retain provider-supported controls such as prompt_cache_key and prompt_cache_retention. Providers that automatically cache eligible repeated prefixes continue to do so through Router. Check a model's router.capabilities.prompt_caching value from GET /v1/models before relying on prompt caching.

Provider caches are scoped to a provider and model. If a routed model or optimization selects a different provider or model on a later request, it cannot reuse the first provider's warm cache. Eligible multi-candidate OpenAI routes carrying prompt_cache_key use a five-minute routing-affinity lease by default, extended to 24 hours for prompt_cache_retention: "24h", while still failing over when the pinned candidate degrades.

Self-service Router response caching, which would reuse an entire previous response without calling a model provider, is a separate optimization and is not currently configurable.