Skip to content

Fall back to another model

The concept behind models and routing is explained in Models and routing.

When the model you asked for is unavailable, Core can run the turn on the next model instead, without you sending the request again. Which model that is comes from a routing configuration your organisation's administrators set up. This page describes what your application sees of it: what your key shows, what does and does not fall back, and what the response says about it. An administrator sets the configuration up on Fallback routing.

What a routing configuration is

A routing configuration is a named list of chains. Each chain names one model and, in order, the one to four models to try instead of it:

{
  "name": "default",
  "chains": [
    { "primary": "gpt-5.6-sol", "fallbacks": ["claude-sonnet-5", "gemini-3.8-flash"] }
  ]
}

An administrator creates it on your organisation and then either makes it the organisation's default — so every key of the organisation uses it — or names it on one key, which overrides the default for that key alone. A key can also be switched out of falling over entirely. Which configuration a key names is a key setting, PATCH /keys/{key_id}, and a key holding management can create, list, read, rename, replace and delete the configuration objects themselves too, and list and read their versions (.../{id}/versions), on /routing-configs for its own organisation and /organisations/{id}/routing-configs for one below it — what /tenants/{id}/routing-configs offers a person, whose create and update also take draft: true for a change no key falls back on until it is published, and what Manage organisations through the API walks through. The console offers the same on Routing.

What your key sees

GET /key names the configuration in force for your key, or null when it falls back to nothing:

{ "routing_config": { "id": "rcf_0KQ2mR...", "name": "default" } }

GET /models carries, for every model you may call, the chain your turns on it actually walk — already narrowed to what your key's policy allows:

{ "id": "gpt-5.6-sol", "object": "model", "fallbacks": ["claude-sonnet-5"] }

An empty fallbacks means a turn on that model will not fall over to anything.

A model on the chain whose retirement day has passed appears as its replacement, or is left out where your organisation's or key's policy blocks retired models. The list shows the chain as a turn walks it right now.

Both show your key's own configuration, because they name no agent. A turn whose agent or request names another configuration falls back along that one (below).

A configuration for one agent or one turn

One key that serves several assistants can give each its own chains. Two places name a configuration besides the key and the organisation:

  • An agent version may carry routing_config_id. Every turn of that version falls back along it instead of the key's.
  • A single POST /responses request may carry it too, for that turn alone:
curl -s -X POST "$CORE_URL/responses" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"agent": {"id": "support"}, "input": "Summarise this week.",
         "routing_config_id": "rcf_..."}'

The nearest one wins as a whole: the request's, then the agent version's, then the key's, then the organisation's default. Chains are never combined. A configuration that is not one of your organisation's with a published version is a 422 on routing_config_id, on the request and on the agent version alike. A key whose fallback is switched off falls back on nothing, whatever the agent or the request names.

A turn on an older agent version whose configuration has since been deleted falls back on nothing, not on the key's configuration.

Creating and assigning configurations in the console is described in Fallback routing.

What falls over, and what does not

A turn moves to the next model when the one it was on could not answer: it timed out, the provider answered 429 or a 5xx, or the connection was never made. Core tries each model the way it always has — retries first, then the next model on the chain.

Three failures stay where they are, because the next model would answer them exactly the same way:

  • a 4xx other than 429, which is something about the request itself;
  • a model that is not in the catalogue at all;
  • a turn that could not hide personal data before sending it.

Two limits are worth knowing:

  • A streamed turn falls over only before its first piece of text. Once you have been sent text, starting again on another model would repeat it, so the turn fails instead.
  • The chain is walked once, left to right, and only the chain of the model you asked for. A fallback's own chain is never consulted, so a configuration can never loop.

What the response tells you

model is the model that actually answered:

{
  "status": "completed",
  "model": "claude-sonnet-5"
}

There is no second field beside it. model in the Responses API means the model that produced the answer rather than the alias you typed, so a turn that fell over says so by naming the model that ran. A turn that failed on every model on the chain names the model you asked for — status is then failed with the error.code of the last model's failure, exactly as it would be with no configuration at all: model_error, model_timeout, or model_rate_limited when the chain ended on a rate limit. A streamed turn carries the same field in the response object inside its response.completed or response.failed event; there is no separate event for a fallover, and the response.created event names the model you asked for, because no round has answered yet.

Once a turn has fallen over, it stays on the model that answered for the rest of its tool rounds — one turn, one voice. Your usage rows are written against the model that actually ran, so what you are billed for and what answered are the same thing.

A background run starts again at the model you asked for each time a worker picks it up: an outage that was happening a minute ago may well be over.