The pipeline document
The pipeline document of a scope sets the stages of its chat calls and the check of the answer. A scope is the project, one app or one key. The document is the section pipeline of the routing document of the scope.
The checked path describes what each stage does.
The format
stages: [intake, draft, verify, escalate]
check:
instructions: "Is every claim of the answer supported by the source?"
threshold: 1.0
The fields
| Field | Type | Default | What it holds |
|---|---|---|---|
stages | list of stage names | [intake, draft] | The stages that a call runs, in order |
check | object | none | The check of the answer, which the stage verify asks |
check.instructions | text | none | What the decision model checks about each claim |
check.threshold | number, 0 to 1 | 1.0 | The share of the claims that must have support |
A document holds no other field. A field that is not in this table is a fault.
The stages
A stage name is one of these values. The list of stages runs in this order, so a list in another order is a fault.
| Stage | What it does |
|---|---|
intake | One call reads the request and answers the questions of the classify document |
context | A retriever keeps the passages that the answer may use |
draft | A model of the tier writes the answer |
verify | The check reads the answer claim by claim |
escalate | The next tier chain answers again, and the check runs again |
The rules of a document
Proxium refuses a document that breaks one of these rules. The fault names the field that holds it.
- The list of stages holds at least one stage.
- The list starts with
intake. - The list holds
draft, because the pipeline answers the request. - No stage is in the list twice.
- Each stage is in the place that the table of the stages gives it.
- A pipeline that runs
verifysetscheck. - A pipeline that holds
checkrunsverify. - A pipeline that runs
escalatealso runsverify, because the escalation follows a failed check. check.instructionsis not empty, and it is at most 2000 characters.check.thresholdis from 0 to 1.- The whole JSON of the document is at most 16000 characters.
The fault of a broken rule starts with its field, for example:
pipeline: stages[2]: 'intake' runs after 'draft', so its place is later
pipeline: check: the stage verify checks the answer, so set "check"
pipeline: check.threshold: 1.5 is not 0 to 1
A claim that the source contradicts fails the check at any threshold. The threshold sets the share of the claims that must have support.
The documents that come with Proxium
| Name | stages | The answer |
|---|---|---|
classic | [intake, draft] | The answer goes to your app with no check. This is the path of today. |
checked | [intake, draft, verify, escalate] | The check reads the answer, and a claim without support escalates the call once. |
A scope with no pipeline section runs classic. classic checks nothing, so its answer never changes.
The result of the check
The check counts each claim in one of three groups: verified, unsupported and contradicted. The support is the number of verified claims divided by the number of claims. An answer with no claim has full support.
A pipeline without the stage verify gives the outcome pass for every answer. A check that did not answer, because of a timeout or an error, gives pass too.
| Outcome | When | What your app gets |
|---|---|---|
pass | No claim failed, and the support is at least the threshold | The answer of the draft |
escalate | A claim failed, and the pipeline runs escalate | The answer of the next tier chain, after a second check |
uncertain | A claim failed after the escalation, or the pipeline has no escalate | The best answer, with the claims that have no support |
Save the document
The document goes with the other sections of the routing document. It has the order of the routes: the app, the key, the project, and then the platform default. Each save is a new version of the whole document. The highest version is in force.
- Read the routing document of the scope, and keep its version.
- Add or change the
pipelinesection. - Plan the change. The answer names each fault with its path, and the sections that change.
- Apply it with the version of step 1. A save with an older version is refused.
The steps and the API routes are in the routing document.