Skip to main content

Plan reuse

An agent calls the model once for each step of its tool loop, and each call reads the whole context again. Many steps are already decided by the step before: after a search, the agent reads the file that the search found. Plan reuse lets the model write that next step down with the tool call before it. When the result of the tool call shows that the planned step applies, Proxium answers with the planned tool call and calls no model. Your agent still runs every tool itself.

The method is from "HarnessMM: Recursive Self-Improvement for Harness Token Efficiency" (arXiv:2610.32813).

How it works​

  1. Each tool of a call gets one more optional argument, proxium_next, and a short instruction goes after the system prompt. With a tool call, the model lists in proxium_next the tool calls that it will make after this one, when their arguments are certain now.
  2. Each entry has a label, the tool, its literal arguments and a predecessor: root for the result of this call, or an earlier entry. Its guard reads the result of the predecessor, so the entries form chains.
  3. Proxium cuts proxium_next out of the tool call before your app sees it, streamed or not. In a stream, Proxium holds each tool call until it is complete and then sends it whole. Your app cannot run a tool before its arguments are complete, so the hold adds no wait.
  4. On the next call, the result of the tool call is in the request. The entries after that step whose guard passes on the result are eligible. Exactly one eligible entry selects it. None, or two, select nothing. When an answer has several tool calls, each call keeps its own proxium_next. The guard of each entry reads only the result of its own call.
  5. In shadow, the call goes to the model as always, and Proxium compares the call that the model makes with the selected entry. Each comparison is one sample of the pair: the tool and the kinds of its guard.
  6. In skip, a selected entry of an open pair is the answer, with no model call. The answer has the header x-proxium-skipped: plan. Its result can open the next entry of the chain, so one model call can stand for several steps.

These guard kinds exist: exit_code, contains, not_contains, max_lines, min_lines and file_exists. All parts of a guard must pass. The gateway cannot see your files, so a file_exists guard never passes there; the local router of Claude Code checks it on your machine.

Which calls can skip​

Only a read can skip: read, grep, glob, ls, list and the reads of the codeindex MCP server. The same tools of Claude Code can skip too: Read, Grep, Glob, LS and mcp__codeindex__…. A planned call that writes, deletes, runs a build or a test, or uses the network never skips. Shadow still measures it, and the report marks it as "never skips".

The gate of each pair​

Each tool and guard pair of each app has a gate:

  • A pair opens when its shadow precision is at least 99 % over at least 500 samples.
  • After a skip, the next model call decides whether the skip was right. It was a regret in three cases. The call makes the same tool call again. The model writes that the last call was not needed or was wrong. The call fails.
  • A pair closes when more than 1 % of its last 200 skips were regrets: 3 regrets in 200 skips close it. A closed pair goes back to shadow, and starts its samples again from none.

The four numbers are settings of each project and app.

The fill gate of each model​

The instruction and the proxium_next argument add input to every call with tools. Some models write a usable plan with their tool calls. Other models almost never write one, so they pay for the instruction and get nothing for it.

So Proxium counts, for each app and each model that answers, the answers with tool calls and the answers that hold a usable plan. A plan is usable when one or more of its entries has a guard. An entry with no guard passes on every result, so it cannot choose a step.

  • After a window of answers, Proxium compares the share of usable plans with the fill rate of the app.
  • Below the fill rate, the gate of that model closes. Its calls go out with no instruction and no proxium_next, so the model pays nothing for plan reuse.
  • After a number of answers with the gate closed, Proxium sends the instruction again for one more window. A new version of the model, or a better instruction, can then come back.
  • When the app asks for a route or a tier, the gate is the one of the model that answered that name last time.

The three numbers are settings of each project and app: min_fill_bp (the fill rate, in basis points), fill_window and fill_retry. A fill rate of 0 never closes the gate. A reset of the gates of a whole app also measures its models again.

The local router of Claude Code has the same gate, with its state on your machine.

Turn it on​

In the console, open Requests. The Plan reuse part has the switch for every app of the project: off, shadow or skip.

With the MCP server:

proxium_set_plan_reuse  source: "*"         mode: "shadow"
proxium_set_plan_reuse  source: "opencode"  mode: "skip"   min_samples: 500  min_precision_bp: 9900
proxium_set_plan_reuse  source: "opencode"  mode: "shadow" min_fill_bp: 2000  fill_window: 50  fill_retry: 500
proxium_reset_plan_gates  source: "opencode"  pair: "read|contains"

The row of an app wins over the row of every app (*). A project with no row has plan reuse off. Off removes the instruction from the calls at once, so no plan and no skip follow.

Start with shadow. Turn on skip when the report shows the pairs that you want to skip at 99 % or more.

Read the report​

The Plan reuse part of Requests, and proxium_read_plan_reuse, show for the time window:

NumberWhat it means
PlansThe answers with tool calls that held a plan, of all the answers with tool calls that got the instruction
Banks filledFor each app and model, the answers that held a usable plan, the share in the last window, and whether the gate still sends the instruction to that model
Shadow precisionOf the samples, the share where the model made the selected call
Gateshadow, with the precision and samples since the pair last closed, or open, with the regrets in its last skips
SkipsThe calls that Proxium answered with no model call
RegretsThe skips that the next model call showed as wrong, and their share of the skips with an outcome
SavedFor each skip, the input, the cached input and the output of the model call before it, which the skip stands for, and its cost
Wait savedThe time that model call took, added up over the skips
Plans costThe output tokens of the plan blocks, about 4 characters for each token, and their price on the model that wrote them
Tokens and money saved, netWhat the skips saved, less what the plan blocks cost

The plan blocks cost output on every call with tools, and the skips save whole calls. The net numbers show whether the trade pays for your traffic.

The instruction in force​

The instruction has a version. Once a week, the gateway scores changes of it on the recorded steps of the projects in shadow or skip. A change offers one guard kind fewer, or asks for one action fewer. The job makes no model call: it scores each recorded plan as the changed instruction would have let it be.

It keeps a change only when the precision stays at 99 % or more and the tokens fall by 10 % or more. The kept change becomes the next version, and the report names the version in force. The prompt cache keeps its prefix while the version stays the same. The local router of Claude Code keeps version 1.

A recorded step holds the plan and the first 8,000 characters of the tool result. It also holds the call that the model then made, and the tokens of that call. Proxium keeps the newest 2,000 steps of each project for 8 days, and only while plan reuse is on.