Skip to main content

Debug a failed call

When a call fails, first find out who refused it: Proxium, before any vendor call, or the vendors that Proxium tried.

1. Read the status and the code​

Status and codeWho refusedVendor calledNext step
401 missing_key, 401 invalid_keyProxiumNoCheck the key. See Errors
400 no_routeProxiumNoRead the message. Your project has no model for this request
429 tenant_capped, 429 source_cappedProxiumNoA ceiling. Wait retry-after seconds. See Set budgets and limits
503 upstream_failedThe vendorsYes, each model of the planDo step 2

The message of 503 upstream_failed gives the reason of the last attempt, for example:

503 body
{
"error": {
"message": "client deadline exceeded",
"type": "upstream_failed",
"code": "upstream_failed"
}
}

2. Find the call on the Requests screen​

  1. In the console, open Requests.
  2. Select the Time window: Today, Last 7 days or Last 30 days.
  3. Optional: to see one app only, select it in the app bar. All apps shows every app.

The log has one row for each request. A request that failed over names each model tried in its Route. Open the row and it lists one block for each attempt, numbered Try 1, Try 2, …. Each block names the model, what you asked for, the outcome and the time.

In this example the first model failed, and Proxium tried the second model of the chain. Your app got the answer of the second model.

OutcomeMeaning
successThe model answered. Reason empty: finish_reason=length means that the vendor cut the answer at your max_tokens before any content. Proxium retries on the same model at the level low, then on the next models. Each cut answer is billed and has a usage row. When the retries end, your app gets the last answer and the header x-proxium-warning. Raise max_tokens: a reasoning model can spend the whole budget before it writes
emptyThe model answered 200 with no content and no tool call. Proxium skipped that model for the request, and tried another model of the same vendor when there was one
errorThe model answered with an error, or the connection failed
timeoutThe model did not answer in time
circuit_openProxium skipped the vendor, because its circuit breaker was open. See Routing
context_skippedProxium did not call the model, because the prompt is longer than its context window. Reason gives the estimate and the window. A row with the reason ContextExceeded is a model that refused the prompt for its length
at_capacityProxium did not call the vendor, because the vendor had as many calls in flight as its cap, and tried the next model of the chain. The cap holds for the platform key of the vendor. A call with your own key has no cap. See Routing
note

An image, speech or video call has one row, with Try 1: these routes do not fail over. A video that fails while it renders gets a second row, with the reason video_failed, when your app polls it. The drill-down of a media call shows the request as JSON, and the error text of a failed call. It never shows the image, the audio or the video.

3. Read what went out and what came back​

Select a row of the log. The row opens, with each attempt of the request.

Each attempt starts with a strip: the outcome, the try, the model, the tier that you asked for, the finish reason and the tokens.

You seeIt means
finish stopThe model finished its answer
finish lengthThe model reached its output limit
finish tool_callsThe model asked your app to run a tool
"No answer text: the model stopped at its output limit…"The model used all its output tokens to reason, and wrote no answer. Raise max_tokens

For a failed attempt, Came back comes first: it is what the vendor answered. Sent upstream shows the request as messages. The messages that earlier calls sent are one folded row. The messages new in this call are open.

To see the exact JSON, select Raw. Then Copy request and Copy answer copy it.

warning

The row shows a prompt only if the project stored it. A new project stores the prompts of failed attempts only. Step 4 shows how to change that.

4. Keep the prompts that you need​

The setting What to store, in Settings › Stored prompts and answers, decides what the drill-down can show:

ValueThe drill-down shows
offNo prompt and no answer
errors (default)The prompt and the answer of each failed attempt
allThe prompt and the answer of each attempt

Any member can change it. Data handling says how long Proxium keeps them.

5. Find a pattern​

To see if one vendor or one model fails often, read the panels of Performance:

PanelIt shows
Failures by reasonThe failed attempts, by outcome, vendor and reason, with a count
Failures by providerEach vendor, with its share of the failures and the kinds of failure
Models and vendorsEach vendor and each model: answered calls, the share answered, tokens, cost, the failures (429, 410 Gone, other) and the answer time. Retired now lists the models and the vendors that your project skips, with retired until a time and the reason

Your failover policy retires a model that fails on calls in a row. It also retires a vendor that fails across its models, or that answers that your own key has no credits. The next model of the plan answers. The reason in Retired now tells you what to fix:

ReasonFix
The vendor does not list this name (404)Fix the model name in your code, or in the tier
The vendor retired this model (410)Remove the model from the tier, or replace it
The vendor answered that your key has no credits (402)Add credits at the vendor
The vendor answered that a daily, weekly or monthly limit or a free tier is used upWait for the vendor's reset; Proxium skips the vendor until then. To use it before the reset, add credits at the vendor, then select Bring back
The vendor rejected your key (401)Replace the key on Providers
Failed calls in a row, last 5xx or a connection errorWait for the vendor, or move the model down in the tier

To call a retired model or vendor again before its time ends, select Bring back. If one model fails often, put a different model first in the chain. See Retire a model or a vendor.