The checked path
Proxium has two paths for a chat call. The request flow is the path of today, which routes the call and returns the answer of the model. The checked path keeps that routing and adds a check of the answer. A claim without support goes to a second model, and a second check follows.
A scope sets its own path with a pipeline document. A scope is the project, an app or a key. The document names the stages of the path and the check of the answer.
The stages of the checked path
A call through the checked path passes these stages:
- Guard. The scan for sensitive data runs as it does today, with the format scan and the AI checks. See Sensitive data scanning.
- Intake. One call to the decision model reads the request and answers every question of the classify document of the scope. The answers choose the tier, the reasoning level and the first model. See Automatic routing.
- Context. A retriever keeps the passages that the answer may use. The decision model judges each passage. It drops a passage with injection or with no relevance, and it marks a passage that contradicts the source.
- Draft. A model of the tier writes the answer, from the request and the kept passages.
- Verify. The check reads the answer claim by claim. A claim that the source supports is verified. A claim that the source does not support is a failed claim. A claim that the source contradicts fails the check.
- Escalate. When a claim failed, the next tier chain answers again, with three inputs: the request, the draft and the failed claims. The check then reads the new answer.
- Uncertain. When the escalated answer also fails, the answer goes to your app with the claims that have no support.
Every stage writes one row to the trace of the request. The row names the stage, its outcome and the answers that the decision model gave.
The pipeline document
The pipeline document of a scope sets the path of its calls:
stages: [intake, draft, verify, escalate]
check:
instructions: "Is every claim of the answer supported by the source?"
threshold: 1.0
stages lists the stages that a call runs, in order. check holds the question that the stage verify asks, and the share of the claims that must have support. The pipeline document is the full reference.
The document is one section of the routing document of the scope. So it has the order of the routes: the app, the key, the project, and then the default of the platform. Proxium checks the document on save and keeps each version, as it does for the sections of the routing.
Two documents come with Proxium, and a scope names one of them:
| Name | Stages | The answer |
|---|---|---|
classic | intake, draft | The answer goes to your app as it is. No check runs, so the answer never changes. This is the path of today. |
checked | intake, draft, verify, escalate | The check reads the answer, and a failed claim escalates the call once. |
What the check found
The check of an answer counts each claim in one of three groups: verified, unsupported and contradicted. From those counts Proxium reads the support: the verified claims divided by every claim. An answer with no claim has full support.
The answer passes when two things are true: no claim failed, and the support is at least threshold. The default threshold is 1.0, so every claim needs support. A claim that the source contradicts always fails the check, at any threshold.
From the result of the check Proxium takes one of three outcomes:
| Outcome | When | What your app gets |
|---|---|---|
pass | No claim failed, and the support is at least the threshold | The answer of the draft, with no change |
escalate | A claim failed, the pipeline runs escalate, and no escalation ran before | The answer of the next tier chain, after a second check |
uncertain | A claim failed after the escalation, or the pipeline has no escalate stage | The best answer, with the claims that have no support |
When the check does not answer
The check is a call to the decision model, so it can time out or fail. A check that does not answer passes the answer through, as the guardrail check of the decision model does today. Proxium records the reason in the trace and adds no claim. A decision model that is slow therefore costs the check, not the answer.
What runs today
Proxium stores the pipeline document of a scope today, and it refuses a document that breaks a rule. The section in force is part of the routing document of the scope, with its versions.
The intake of the checked path is the classification profile of the scope, which runs today. It asks its questions in one call to the decision model, as the intake of the checked path does. See Classification documents.
The stages context, draft, verify, escalate and uncertain do not run yet. They wait on the decision engine, which carries the decisions store and the route POST /v1/decisions/{name}. Until then, a scope that names a document with those stages keeps the answer of the path of today. The document is saved, checked and versioned, and it runs when the engine reaches the request path.