> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ironlabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Routing Lifecycle

> The five states every IronLabs request passes through, what triggers each transition, and what you can observe at each step.

Every request to IronLabs follows the same five-state lifecycle. Understanding these states is the difference between debugging "why did it pick that model?" in minutes versus hours.

```mermaid theme={null}
stateDiagram-v2
    [*] --> Received
    Received --> Scored: embed + classify
    Scored --> Routed: top-k ranking
    Routed --> Executed: dispatch to provider
    Executed --> Logged: success
    Executed --> Fallback: provider error
    Fallback --> Executed: retry next in chain
    Fallback --> Logged: chain exhausted
    Logged --> [*]
```

## State-by-state

### Received

**Trigger:** Your SDK or HTTP client sends a `POST /completions` (or `/model-select`) call. The API gateway authenticates the bearer token, validates the payload schema, and assigns a **`responseMessageId`** (UUID).

**Observable:** No content yet — only the request ID. If auth fails or schema is invalid, the lifecycle terminates here with a 4xx and never reaches **Scored**.

### Scored

**Trigger:** The router embeds the user message and runs the classifier (either a **pre-trained router** for general use or your **Custom Router** if `routerId` is set). Each candidate model in `models` gets a score that blends three signals:

| Signal             | Source                                                          |
| ------------------ | --------------------------------------------------------------- |
| **Capability fit** | classifier output — "is this model likely to answer correctly?" |
| **Cost**           | normalized per-million-token price                              |
| **Latency**        | rolling p50 time-to-first-token from the last hour              |

Your `tradeoff` setting (`"latency"`, `"cost"`, `"quality"`) controls the weighting.

**Observable:** Internal — surfaced in the Studio activity log as the **decision trace**.

### Routed

**Trigger:** Top-k ranking is reduced to a single primary plus an ordered fallback list. The system reserves capacity at the chosen provider.

**Observable:** Streaming requests start seeing the chosen provider/model echoed in the first SSE chunk:

```json theme={null}
{ "provider": "anthropic", "model": "claude-3-5-sonnet-20240620", "type": "metadata" }
```

### Executed

**Trigger:** IronLabs forwards the request to the provider with normalized parameters. Streaming responses are pass-through SSE; non-streaming responses are buffered.

**Observable:** Either the full assistant message (non-streaming) or a stream of `text-delta` events (streaming). The `usage` block — `{prompt_tokens, completion_tokens, total_tokens}` — is attached at the close of execution.

### Fallback (optional)

**Trigger:** The provider returned a 5xx, timed out, exceeded the request's `maxTokens`, hit a content-policy block, or violated your `maxRetries` budget. IronLabs walks to the **next entry** in the fallback list and re-enters **Executed**.

**Observable:** The final response includes an `errorTrace` array listing each attempt that failed. Inspect this when you see a slower-than-expected response — most often, a primary timed out and a fallback served.

```json theme={null}
"errorTrace": [
  { "provider": "openai",    "model": "gpt-4o", "error": "timeout (5s)" },
  { "provider": "anthropic", "model": "claude-3-5-sonnet-20240620", "error": null }
]
```

If the entire chain (primaries + fallbacks) is exhausted, the lifecycle exits **Fallback → Logged** with HTTP `502` and an `errorTrace` describing every attempt.

### Logged

**Trigger:** Whether the request succeeded or failed, IronLabs writes a single row to `ModelSelectUsage` with: `userId`, `customRouterId` (if any), `provider`, `model`, `cost`, `latency_ms`, `errorTrace`, `responseMessageId`.

**Observable:** Visible in the **Studio → Activity** tab within \~10 seconds. Cost rolls up into your **Balance** via `DebitTransaction`.

## Edge cases

* **`previous_session` for context continuity.** When set, the router prefers the **same model** that handled the prior turn — `Scored` is short-circuited to maintain conversation coherence. The trace records `routingReason: "session_continuity"`.
* **`stream: false`** waits for full execution before returning. Network round-trip is faster but you lose progressive output.
* **No fallback configured.** `Fallback → Executed` is skipped — a single provider failure surfaces as `502` immediately. Set `fallback_models` for production traffic.
* **Custom Router cold start.** First request after 5 minutes idle loads the XGBoost classifier from Modal Volume; expect +200–500ms on the cold call. Subsequent calls are warm.

## Related

<CardGroup cols={2}>
  <Card title="Fallback Configuration" icon="shield-halved" href="/refinery/essentials/model-select">
    How to set primaries vs fallbacks.
  </Card>

  <Card title="Custom Router" icon="route" href="/refinery/essentials/custom_router">
    Replace the pre-trained scorer with your own.
  </Card>

  <Card title="Completions API" icon="code" href="/refinery/api-reference/endpoint/completions">
    Every parameter that affects routing.
  </Card>

  <Card title="Sandbox Isolation" icon="lock" href="/refinery/essentials/sandbox-isolation">
    How tenant data stays separated across requests.
  </Card>
</CardGroup>