Shadow models

Stop overpaying for every request. Find the cheapest model that performs on your real production traffic.

Shadow models let you evaluate alternative models against the model serving production traffic before you change your routing configuration. Router mirrors a percentage of eligible POST /v1/responses requests to one to three models in the background.

The routed model remains the primary model. Its response is the only response returned to your application, and its normal billing behavior does not change. Shadow responses are recorded for analysis, are not returned to the caller, and do not affect routing or primary-request latency. Shadow provider calls are not billed to you.

Configure shadow traffic

On the Strategies page, select one to three shadow models and the percentage of eligible traffic to mirror. Request content recording must be enabled so Router can evaluate the primary and shadow outputs.

Each eligible request is sampled independently. When Router selects a request, it creates a shadow run containing the primary response and one response from each configured shadow model.

Shadow execution is detached and bounded. A failure, timeout, or slow response from a shadow model does not change the response returned to your application or cause the primary request to fail.

Compare results

Router evaluates each shadow model across three dimensions:

Dimension	How it is evaluated
Cost	Direct comparison of each model's recorded cost for the same request. Shadow calls remain non-billable, but their provider list cost is retained for comparison.
Latency	Direct comparison of provider latency for the primary and shadow calls. Shadow latency is measured independently and never delays the primary response.
Agreement (coming soon)	An LLM judge compares each shadow response with the primary response and determines whether the outputs align.

Open Shadow Logs to inspect paired responses and compare aggregate cost and latency over time.