Skip to main content

The pipeline document

The pipeline document of a scope sets the stages of its chat calls and the check of the answer. A scope is the project, one app or one key. The document is the section pipeline of the routing document of the scope.

The checked path describes what each stage does.

The format​

stages: [intake, draft, verify, escalate]
check:
instructions: "Is every claim of the answer supported by the source?"
threshold: 1.0

The fields​

FieldTypeDefaultWhat it holds
stageslist of stage names[intake, draft]The stages that a call runs, in order
checkobjectnoneThe check of the answer, which the stage verify asks
check.instructionstextnoneWhat the decision model checks about each claim
check.thresholdnumber, 0 to 11.0The share of the claims that must have support

A document holds no other field. A field that is not in this table is a fault.

The stages​

A stage name is one of these values. The list of stages runs in this order, so a list in another order is a fault.

StageWhat it does
intakeOne call reads the request and answers the questions of the classify document
contextA retriever keeps the passages that the answer may use
draftA model of the tier writes the answer
verifyThe check reads the answer claim by claim
escalateThe next tier chain answers again, and the check runs again

The rules of a document​

Proxium refuses a document that breaks one of these rules. The fault names the field that holds it.

  1. The list of stages holds at least one stage.
  2. The list starts with intake.
  3. The list holds draft, because the pipeline answers the request.
  4. No stage is in the list twice.
  5. Each stage is in the place that the table of the stages gives it.
  6. A pipeline that runs verify sets check.
  7. A pipeline that holds check runs verify.
  8. A pipeline that runs escalate also runs verify, because the escalation follows a failed check.
  9. check.instructions is not empty, and it is at most 2000 characters.
  10. check.threshold is from 0 to 1.
  11. The whole JSON of the document is at most 16000 characters.

The fault of a broken rule starts with its field, for example:

pipeline: stages[2]: 'intake' runs after 'draft', so its place is later
pipeline: check: the stage verify checks the answer, so set "check"
pipeline: check.threshold: 1.5 is not 0 to 1

A claim that the source contradicts fails the check at any threshold. The threshold sets the share of the claims that must have support.

The documents that come with Proxium​

NamestagesThe answer
classic[intake, draft]The answer goes to your app with no check. This is the path of today.
checked[intake, draft, verify, escalate]The check reads the answer, and a claim without support escalates the call once.

A scope with no pipeline section runs classic. classic checks nothing, so its answer never changes.

The result of the check​

The check counts each claim in one of three groups: verified, unsupported and contradicted. The support is the number of verified claims divided by the number of claims. An answer with no claim has full support.

A pipeline without the stage verify gives the outcome pass for every answer. A check that did not answer, because of a timeout or an error, gives pass too.

OutcomeWhenWhat your app gets
passNo claim failed, and the support is at least the thresholdThe answer of the draft
escalateA claim failed, and the pipeline runs escalateThe answer of the next tier chain, after a second check
uncertainA claim failed after the escalation, or the pipeline has no escalateThe best answer, with the claims that have no support

Save the document​

The document goes with the other sections of the routing document. It has the order of the routes: the app, the key, the project, and then the platform default. Each save is a new version of the whole document. The highest version is in force.

  1. Read the routing document of the scope, and keep its version.
  2. Add or change the pipeline section.
  3. Plan the change. The answer names each fault with its path, and the sections that change.
  4. Apply it with the version of step 1. A save with an older version is refused.

The steps and the API routes are in the routing document.