Step classifier

The step classifier labels every inbound request with the agent-loop step it represents, so the gateway can route each step to the model that historically wins on that kind of work. A planning step and a tool-arg formatting step have different cost and quality trade-offs; pinning one model for both overpays on the cheap step or underperforms on the hard step. The classifier names the step, and the router plus the capability resolver bias the model pick toward the right tier for that step.

The classifier is per-step routing intelligence. It runs on the prompt text of the last user message and emits a step-class label, a complexity tier, and a thinking budget. The labels are descriptive of the work, not the model tier, so the vocabulary stays stable even as the underlying model picks change.

The step-class vocabulary

The classifier emits one of six step-class labels:

The labels are mutually exclusive; the classifier picks the single label that best fits the step, favouring the most agent-loop-specific signal. When no label clearly applies, the classifier returns no step-class and the request routes on its complexity tier alone.

Complexity tier and thinking budget

Alongside the step-class label, the classifier emits a complexity tier of simple, medium, or complex, and a thinking budget of none, low, medium, or high. The complexity tier maps to a model family: simple to a cheap fast model, medium to a mid model, complex to the operator’s chosen flagship. The thinking budget controls how much extended reasoning the model spends on the call, where none runs the model with no extended thinking, low buys a cheap nudge, medium sits between a nudge and a full reasoning pass, and high buys the long-form chain-of-thought a hard planning or synthesis step needs. Each setting maps onto whatever the destination provider actually supports, so the same tier becomes an Anthropic thinking budget, an OpenAI reasoning effort, or a Gemini thinking budget without your call site changing.

The budget can also be resolved for you. With the org-level auto mode on, the gateway’s complexity judge picks the tier per request rather than the caller pinning one, and every request records where its budget came from, whether the client asked for it, the org default applied, or auto derived it, so the choice is auditable on the usage page.

The two axes are independent. A short planner prompt can carry a high thinking budget on a medium complexity tier, because the value is in the reasoning depth rather than the raw model size; a long tool-arg prompt can carry a thinking budget of none because the output shape is the function call and there is nothing for the model to think about.

How the step class biases routing

The step-class label biases the model pick after the complexity tier and the thinking budget are set. On a request that names a capability alias, a step labelled planner is lifted to the heavy-reasoning mapping with an extended thinking budget, because the planning step rewards the long-form reasoning path. A step labelled tool-arg-formatter drops to the cheap-fast mapping with the thinking budget off and JSON mode pinned on the downstream request, because the schema shape is the output and there is nothing for the model to think about. A step labelled critic keeps its model but has to run on a provider other than the one that produced the step it is reviewing, so a model never grades its own family’s output.

On the plain auto path the bias is narrower and opt-in per org. With it on, three low-blast-radius steps — tool-arg-formatter, summariser, and extractor — route one tier cheaper than the classifier’s verdict, because a weaker summary or a missed extraction field is cheap to absorb. The load-bearing steps — planner, coder, and critic — stay on the tier the classifier picked: their output steers everything after them, so a cheap-model regression there has a wide blast radius.

The bias composes with the capability resolver’s focus-mode bias rather than overriding it. When an operator flips the org’s focus mode to cost reduction, the resolver prefers the cheaper variant within the step’s tier; when the operator flips to quality mode, the resolver prefers the premium variant. The step-class bias and the focus-mode bias stack, so the final model reflects both the kind of step and the operator’s cost posture.

Latency and replay

The step classifier runs in the request’s fast path. It adds no extra network round-trip and stays off the request’s latency budget, so the labelling cost is negligible compared to the upstream model call. The label it returns is persisted to the request’s routing record, so the session trajectory can replay the step class the gateway saw for every call in the loop.