Use the Responses, embeddings, rerank and moderations APIs
Proxium serves five more OpenAI APIs with the same base URL and virtual key as chat:
| Route | What it does | Routing type | Stream |
|---|---|---|---|
POST /v1/completions | The legacy text completion API | Chat | Yes |
POST /v1/embeddings | Turns text into vectors | Embeddings | No |
POST /v1/responses | The OpenAI Responses API | Text | Yes |
POST /v1/rerank | Orders documents by how well they match a query | Rerank | No |
POST /v1/moderations | Checks text against a content policy | Moderation | No |
Proxium sends your request body to the same path at the vendor, for example <base URL of the vendor>/embeddings. It sends the answer of the vendor back without change. Jev answers rerank and moderations calls too. See Rerank and moderations.
Each route goes only to a model of its routing type. A rerank call never goes to a chat model, and a moderations call never goes to a rerank model. When the project has no model of the type, the call gets 400 no_route at once.
Make embeddings
- Add a vendor that serves embeddings, for example the OpenAI (embeddings) preset. See Add vendors and your own keys.
- Get the model id from
GET /v1/models, for examplet.acme.openai-embed/<model>. - Send the id in
model:
- Python
- curl
result = client.embeddings.create(
model="t.acme.openai-embed/<model>",
input=["The first text.", "The second text."],
)
print(len(result.data), len(result.data[0].embedding))
curl https://proxium.tech/v1/embeddings \
-H "Authorization: Bearer $PROXIUM_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "t.acme.openai-embed/<model>", "input": ["The first text."]}'
Proxium does not store or search the vectors. It records the tokens and the cost of the call.
To send a tier name in place of the id, set a chain for the tier embed in Routing › Embeddings routing.
Use the Responses API
Send the call to a vendor that serves the Responses API, such as OpenAI. Set stream to true to get server-sent events.
response = client.responses.create(
model="t.acme.openai/<model>",
input="Write one sentence about queues.",
)
print(response.output_text)
Rerank and moderations
Jev, the classifier that Proxium routes with, answers both routes. Jev reads the text and scores each question with a probability:
| Route | What Jev scores | The answer |
|---|---|---|
/v1/moderations | Each category of the OpenAI moderation API, for each input | The OpenAI moderation shape: flagged, categories, category_scores |
/v1/rerank | How well each document answers the query | The shape of Cohere and Jina: results with index and relevance_score, best first |
A project with the pooled vendors of Proxium can use the Jev of Proxium. A project can also add its own TypeSafe vendor. The tiers moderation and rerank start with Jev, and a moderation with no model uses the tier moderation:
result = client.moderations.create(input="Text to check.")
print(result.results[0].flagged)
curl https://proxium.tech/v1/rerank \
-H "Authorization: Bearer $PROXIUM_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "rerank", "query": "refund policy", "documents": ["Refunds take 5 days.", "We ship worldwide."], "top_n": 1}'
Jev reads text only. A moderation with an image part gets 400 bad_request.
A moderation takes at most 256 inputs. A request with more gets 400 too_many_inputs, and Jev makes no call. Each input is one Jev call, and the request takes one call slot for each input. Jev reads at most 24,000 characters of an input. A longer input keeps its start and its end, with a line that holds … between them. Its result carries "input_truncated": true.
You can also use a rerank or moderation model of your own vendor. Add the vendor with the request type Rerank or Moderation. Then send its id in model, or put it in the chain of the tier. Proxium sends your body to the vendor without change.
The answer of a rerank is read in one shape. Cohere and Jina answer with results, and Proxium returns it as it is. Voyage answers with data, the same list under another name. Proxium copies data to results and keeps the rest of the answer. A moderation answer is returned as the vendor sent it.
Each call is priced by the way its vendor bills:
| Vendor bills | How Proxium prices the call | What the model needs |
|---|---|---|
Search units, like Cohere (meta.billed_units.search_units) | Units times the price per 1,000 units | The rerank price per 1,000 units. Without it the call is recorded as a missing price, not as free |
Tokens, like Jina and Voyage (usage.total_tokens) | The tokens as input, at the input price | The input price per million tokens |
| Nothing, like the OpenAI moderation API | No charge | Nothing. The call is recorded at zero |
curl https://proxium.tech/v1/rerank \
-H "Authorization: Bearer $PROXIUM_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "t.acme.my-reranker/<model>", "query": "refund policy", "documents": ["Refunds take 5 days.", "We ship worldwide."]}'
What is the same as chat, and what is not
| Chat and Messages | These four routes | |
|---|---|---|
| Virtual key, ceilings, cost record | Yes | Yes |
| Routing and failover | Yes | Yes |
| Row in the log of Requests | Yes | Yes |
| Response cache | Yes | No. Each call goes to a vendor |
x-proxium-memory: recall | Adds memories | No effect. /v1/responses keeps the call in memory as write |
A call without model | Uses auto: the classifier chooses the tier, or the call uses standard | 400 missing_model. A moderation uses the tier moderation |
A call with model: "auto" | The classifier chooses the tier, or the call uses standard | /v1/responses uses standard, rerank uses rerank, and moderations uses moderation. They have no classifier |
Related pages
- Route requests to models: set the chain of a tier.
- API reference: the body of each route.