Skip to main content

Automatic routing

With automatic routing, your app sends model: "auto", and Proxium chooses the tier for each call. A short greeting can go to a cheap tier, and a long coding task to a strong tier. Your app does not decide.

:::info State on proxium.tech Automatic routing is off on proxium.tech today. A call with model: "auto" uses the models of the tier auto, like any other tier. You can set those models on Routing. :::

How it works​

Fig. 1 · how model auto chooses a tier
off on proxium.techmodel "auto"your callclassifierlast user message≤ 24,000 charstrivialstandardheavycodemodelsof heavyfails, or no user messagetier standardexample: a long coding task → heavyoff on proxium.techmodel "auto"classifierlast user message, ≤ 24k charsnames one tierheavytrivial, standard, heavy or codemodels of heavyfails, or no user messagetier standardexample: a long coding task

The classifier never makes a call fail. If it fails, or the call has no user message, Proxium uses the tier standard.

Two kinds of classifier​

KindWhat it getsWhat it answers
A chat modelA prompt that asks for one word, a tier nameThe tier name
Jev, from TypeSafeThe user text, and the tiers with a description of eachA tier and a confidence from 0 to 1

The classifier tier can hold both kinds. The kind that comes first in the tier runs first. If it fails, or Jev is less sure than the minimum confidence of the gateway, the other kind decides.

What it costs​

The classifier call is a real model call. Proxium records its cost against your project, like any call. A key that reached one of its limits skips the classifier, because Proxium refuses that call anyway.