Skip to main content

The model

Prism sends every question to one model on your Agent Router gateway. Which model that is, is a value you set — and it takes an arbitrary model id, because different Routers carry different catalogues and ours is not yours.

Setting it​

router.defaultModel is the lever:

helm upgrade prism <chart> -n <namespace> -f <your values file> \
  --set router.defaultModel=<the model id your gateway serves>

It applies on the next question. No reinstall, no image rebuild, nothing else to change.

Why this matters more than it looks​

A model your gateway does not serve fails in the most misleading way available: every question fails while the entire rest of the install looks healthy. Pods running, database connected, MCP servers answering, self test green on every other line. Nothing in the infrastructure is wrong, so nothing in the infrastructure reports a problem.

The self test has a model catalog line specifically for this. It reports how many models your gateway serves and whether the one Prism is configured to call is among them — so this failure names itself rather than presenting as "the chat is broken".

It reports the count only. The model ids in your catalogue are your configuration, and they are never recorded in the report or the downloadable bundle. The one model id that does appear is the one Prism is configured to send, which is your setting, not your catalogue.

The model is also how fast an answer arrives​

The model is not only a correctness choice. It is the main lever you have over how long an answer takes — and once the ingress timeout is ruled out, it is very nearly the only one.

Where the time in a turn goes, measured on our own deployment: almost all of it is the model thinking and streaming, not Prism assembling. The two things an operator might reach for first do not move it. Prompt caching is a cost lever rather than a latency one — a cached call measured no faster than an uncached one — and the tool calls Prism makes against your own mirrored data return in milliseconds. So there is no queue to shorten and no index to warm: what remains is model time, and time-to-first-token differs materially between one model and another, and between providers behind the same gateway.

That makes it the cheapest experiment available to you:

  • It applies on the next question — no reinstall, no re-ingest, nothing to rebuild.
  • It is undone the same way, so trying a faster model costs nothing if the answers get worse.
  • The self test's model catalog line tells you immediately whether the id you tried is one your gateway actually serves.

The catalogue is not ours, and it may not hold a faster option. Which models your gateway serves is your Agent Router team's decision, not a Prism setting — so if nothing in the list is quicker, enabling a faster model is a conversation with whoever runs your Router, and that conversation is the remaining lever rather than anything in this chart.

One thing worth saying to whoever asks "why is it slow?": a long turn is often a good one. The longest turn we have measured (6 min 11 s) made 25 tool calls across three sources. A faster model that reasons less is not automatically a better deal — compare the answers, not just the clocks.

Pinning Prism separately​

There is a second, narrower value: app.agentModel.

  • router.defaultModel sets the model for everything on the agent runner.
  • app.agentModel pins Prism alone to a different model, overriding the above for this application only.

app.agentModel: "" — the default — means "follow router.defaultModel", which is what you almost always want. Set it only if you are running Prism alongside other agents on the same runner and want Prism on a different model.

Both are picked up on the next chat turn, including on an install that has been running for months. Clearing app.agentModel back to "" genuinely unpins Prism and returns it to router.defaultModel — it does not leave the old model in place.

Checking what your gateway serves​

The self test's model catalog line is the supported answer, and needs no credentials of your own.

If you want the raw list, it is the standard OpenAI-compatible catalogue endpoint on your gateway, and your own inference key will read it:

curl -H "authorization: Bearer <your inference key>" <your gateway>/models