Skip to main content

Automatic routing

With automatic routing, your app sends model: "auto", or sends no model, and Proxium chooses the tier for each call. A short greeting can go to a cheap tier, and a long coding task to a strong tier. Your app does not decide.

auto has no models of its own. Proxium classifies the call, or it uses the tier standard.

When Proxium classifies​

Proxium classifies a call for auto when the project can reach a classifier. A project has a classifier in one of these cases:

The project hasThe classifier is
A preset (see Ready-made routing)Your own Jev if you added a TypeSafe vendor, then a small model of your vendor
Its own vendor in the classifier tier, set on RoutingThat vendor
The grant of the classifier of Proxium, from the adminThe Jev of Proxium, then a small model of Proxium

A project with none of these does not classify. The console setting platform_classifier turns the classifier of Proxium off for every project that reaches it. A project's own classifier keeps running. Each call for auto uses the tier standard, and Proxium makes no classifier call.

A project with the grant: every text tier​

A project can have the grant of the classifier of Proxium. In that project, the classifier also chooses the tier of a call that names a text tier, such as standard or heavy. So the classifier routes every text call of the project, whatever tier the app sends. If the classifier gives no answer, the call keeps the tier that it named.

The classifier does not change these calls:

  • A call that names a model, such as ollama/glm-5.3-flash. The call gets that model.
  • A call for vision, and a call that sends an image, audio or video. The classifier chooses from text tiers only.
  • Embeddings, and the calls of image, speech and video.

:::info State on proxium.tech The classifier of Proxium is not free for customers. The admin grants it to one project, or to every project of a plan. Without the grant, a project classifies only with its own vendor. :::

How it works​

Fig. 1 · how model auto chooses a tier

For a project with a classifier:

  1. Your app sends model: "auto", or no model.
  2. Proxium sends the last user message to the classifier. For a long message, it sends the start and the end.
  3. The classifier names one tier of the preset: trivial, standard, high, heavy, code or free. Example: a long coding task gets heavy.
  4. Proxium calls the models of that tier, with the normal rules: app, key, project, default.

If the classifier fails, or the call has no user message, Proxium uses the tier standard.

The classifier never makes a call fail. Proxium uses the tier standard in each of these cases:

  • The project has no classifier.
  • The classifier fails, or it is less sure than the minimum confidence and no other step decides.
  • The classifier does not answer within its time limit. A slow first step leaves the rest of the time to the next step. The Requests trace and the Classifier log page show this case as a timeout.
  • The call has no user message.
  • The key or the project is at one of its limits.
  • The operator turned the classifier off.

What the classifier decides​

When the classifier is a System One model, such as Jev, one classifier call decides more than the tier. The call asks all questions together, so the extra questions add no extra call.

  • The tier. As described above.
  • The reasoning level. The classifier chooses how much the served model thinks: none, low, medium or high. It reads the output budget of the call (max_tokens), so a small budget gets little or no thinking. Proxium sends the level to the models that reason. A reasoning_effort in your request always wins.
  • The model to try first. When the operator turns it on, the classifier chooses the first model of the tier. For each model, it reads whether the model reasons, its price, its context window and its benchmark scores. It also reads what the model did on this gateway in the last day. In a long conversation with the same tier, Proxium keeps the model that served it. So the prompt cache of that model stays in use.
  • The guardrail checks. For a project with the AI secret or the prompt injection guardrail, the same call checks the last user message. A classifier that is a chat model never reads a request before the guardrails do.

A project can limit the price of the models of a call with a cost tier. Refer to x-proxium-cost-tier in Request headers.

Classification documents​

A classification document is the whole setup of the classifier's call for one app, one key or the whole project. Proxium turns it into the call to the decision model, so nothing of the call is fixed. A document holds:

  • What the model reads. Any text or JSON. An object {"$from": "<part>"} in it is replaced by that part of the call. The parts of the message are the last user message, the system message, the recent turns and every message. The other parts are the output budget (max_tokens), the app, the recalled memory and a request header. The option parts are the tiers of the routing, the models of those tiers and the reasoning levels.
  • The questions. Any number of questions of any type: yes or no, a choice from options, a score on levels, or a box in an image. Their instructions and options take text or JSON, and the same $from parts.
  • The rules. Each rule has a condition on the answers and its actions. A condition tests the probability of a yes, the chosen option, the probability of one option, a score or a confidence. It can also test the call itself. The built-in fields are request.subagent, request.app, request.key, request.repo, request.conversation, request.task, request.turns, request.tool_turns, request.context_tokens, request.stored_hop and request.stored_hop_age. A document adds its own facts in facts, each with a provider: header, first_header, count, contains, call or store. Its identity names the facts that name a conversation. Jev reads any fact with {"$from": "fact", "name": "..."}, or every fact with {"$from": "facts"}. The models source takes tiers, match and with, so a document chooses the hops that are options and the fields of each one. The built-in fields of a hop are provider, model, reasons, price, cache_read_price, context_window, tiers, health, scores and holds_conversation. A document adds its own fields in model_fields, each with a provider. The provider hop copies a built-in field under a new name. The provider metric gives a counter of the hop over the last window_s seconds, 60 to 86400: calls, failed, cut, median_ms or conversations. conversations counts the conversations that the hop answered, from the live store. The other metrics come from the attempt log, which each pod reads every five minutes. For example, {"name": "open", "from": {"metric": "conversations", "window_s": 1800}} and "with": ["holds_conversation", "open"] let Jev keep the vendor of a conversation and put a new conversation on the vendor with the fewest. A document can choose which memories go into a call. A noul question with "for_each": {"$from": "memory_candidates"} is asked once for each memory found, up to 16. Each question gets its memory as item of its instructions. The rule action "memory_items": {"$answer": "<question>", "min": 0.5, "max_items": 6} then adds only the memories whose yes is at least min, the most probable first. With such a rule, a call with x-proxium-memory: recall gets the choice, not the whole recall. The search for the candidates uses words only, with no embedding call, because Jev judges each one. The next calls of the same user message get the same memory again, so the vendor prompt cache keeps its prefix. A document's placement lists steps that order and filter the hops of the plan. The strategies are stored_hop, spread, exclude, prefer and avoid. Each step can have a when condition, in the grammar of the rules. A document's failover orders the hops after the first one when it fails. It lists groups of equivalent hops, the same model at other vendors, and the keys of the order: equivalent, price or chain. So a rule can send the subagents of Claude Code to another tier, or send one app or one repository to its own model. It combines tests with all, any or not. An action sends the call to a tier, puts a model first, or sets the reasoning level. An action can also add the project's memories, keep the model that holds the conversation's prompt cache, flag the call, or block it. The actions keep and keep_model choose how Proxium shortens the long tool results of the call. keep is off, cut, extract, summarise or facts, and keep_model is the helper model of the last three, as a tier or a provider/model. The request header x-proxium-keep wins over a keep action, and a keep action wins over the app setting. See shorten long tool results. A tier, a model or a level can be the answer of a choice question.
  • When to ask the next hop. A condition on the answers, for example a low confidence, after which the next hop of the classifier chain answers.

All questions of a document go in one classifier call. The answers come back to your app in the response header x-proxium-classification.

The most specific document applies: the app, then the key, then the project, then the default of the platform. With no document, the classifier runs the ready-made document classic, which works as described above. Proxium keeps every saved version of a document.

You write a document on the Classification page of the console, through the API (/api/teams/{project}/classifier-profile), or through the MCP tools of the console. A test runs a document on a text and shows the call that was sent, each answer, and what the rules decided. Ready-made documents exist for the tiers (classic), for the model choice, for context and for security.

The section pipeline of the routing document of the same scope sets the stages of its chat calls and the check of the answer. See the checked path.

One decision for each task​

An agent sends one call for each step of its tool loop. Each step repeats the last user message. Proxium keeps the decision of the classifier for that message. A later step with the same message reads the stored tier, and it makes no classifier call.

The decision is keyed on two things:

  • The task. Proxium reads it from the header x-proxium-task-id. Without that header, Proxium uses the system message and the first user message of the conversation.
  • The last user message.

A new user message gets a new decision. Proxium does not store a fallback decision. A stored decision lasts AXON_CLASSIFIER_REUSE_SECS seconds, 1800 by default. Set it to 0 to turn the reuse off.

On Requests, a call that reused a decision shows the classifier as reused.

The report of the classifier counts the tasks that sent x-proxium-task-id. Read it with proxium_read_classifier_report, or on GET /api/teams/{slug}/classifier. The section tasks holds these numbers:

  • tasks: how many tasks made a decided call in the window.
  • decisions: their decided calls, and reused: how many of them reused a decision.
  • classified: the decisions that asked the classifier.
  • classifier_calls: the classifier calls those tasks made. One decision can make more than one classifier call, so this can be more than classified.
  • decisions_per_task, classified_per_task and calls_per_task: the averages for each task. A task of ten steps shows about one classified call.
  • tasks_with_steps: the tasks that made more than one decided call.
  • tasks_changed_tier: the tasks whose calls named more than one tier.

A call with no task id is not counted here.

Ready-made routing​

On Routing, the Ready-made routing section lists a preset for each vendor of the project. Proxium writes nothing until you select Use this preset. A preset holds:

  • the tiers trivial, standard, high, heavy, code, free, with a description of each that the classifier reads
  • the models of each tier, from that vendor, best first
  • the classifier tier: your own Jev if you added a TypeSafe vendor, then a small model of your vendor

A preset exists for these vendors: OpenAI, Anthropic, Google Gemini, Mistral, DeepSeek, Groq, Together AI, OpenRouter, Ollama Cloud and OpenCode Go. Each provider page under Providers shows the models of its preset.

You doProxium does
Select Use this presetWrites the preset of that vendor
Use the mixed preset, with vendors at several providersWrites, for each tier, the first choice of each vendor, cheapest first, then their other models, cheapest first
Use a preset with a TypeSafe vendor in the projectPuts your own Jev first in the classifier tier
Set the models of a tier yourselfKeeps your models. A preset never writes a tier that you set
Select Copy and editMakes the models of the preset your own. A new version of the preset no longer changes them
Select Reset to the presetWrites every tier of the preset, the tiers that you set included

When Proxium releases a new version of a preset, each project on that preset gets it at the next start of the gateway. A project that selected Copy and edit keeps its models.

A project that uses no preset keeps its own routing. It classifies auto only with its own vendor in the classifier tier, or with the grant. Otherwise auto uses the tier standard.

The classifier of a preset​

You bring the classifier. Proxium suggests one of two kinds:

StepModelKey
1Jev, from TypeSafe, a model made to classify. Add it under Providers as the vendor TypeSafe (Jev classifier)Your TypeSafe key
2A small model of your vendorYour vendor key

Jev decides when it is sure enough. Otherwise step 2 decides. If both fail, Proxium uses the tier standard. The classifier never tries a large model of your vendor.

proxium.tech does not offer its own Jev to customers by default. The admin can grant it to one project, or to every project of a plan.

Two kinds of classifier​

KindWhat it getsWhat it answers
A chat modelA prompt that asks for one word, a tier nameThe tier name
Jev, from TypeSafeThe user text, and the tiers with a description of eachA tier and a confidence

The classifier tier can hold both kinds. The kind that comes first in the tier runs first. If it fails, or Jev is less sure than the minimum confidence, the other kind decides. Each preset sets its own minimum confidence. A project with no preset uses the minimum confidence of the gateway.

What the classifier saved​

For each classified call, Proxium records what the same tokens would have cost on the tier that you named. For auto, that tier is standard. The report of the classifier, GET /api/teams/{slug}/classifier/report?days=, adds up these costs for each tier that served traffic. It also counts the decisions with a confidence under each level from 0.5 to 0.8. A minimum confidence at one of those levels would stop those decisions.

The Classifier page of the console shows the same figures. Pick a window: today, 7 days or 30 days. For each tier, it shows what the calls cost. It also shows what the same tokens cost on the tier that you named. Last, it shows the decisions under each confidence level.

See why a model answered​

On Requests, the Asked for column of an auto call shows the tier that the classifier chose, for example auto → code. When Jev chose the tier, the column also shows the confidence of Jev: auto → heavy (90%).

What it costs​

The classifier call is a real model call. Proxium records its cost against your project, like any call. A key that reached one of its limits skips the classifier, because Proxium refuses that call anyway.