Model Switch and Prompt Cache

A /model switch is a route change inside the live agent — not a new conversation.

What happens on /model

Component	Behavior
Conversation history	Kept and re-sent to the new model
Model / provider / endpoint / API mode / credentials	Re-resolved
Hermes system-prompt cache	Invalidated (agent.switch_model()), rebuilt on the next turn
Compression thresholds	Re-resolved for the new model window
Session id, transcript, goals, metadata	Unchanged

The next request can therefore be heavy from the new provider’s point of view:

new system prompt + kept history + tool results + tools + new user message

For Hermes it is session continuity; for the new provider it is potentially an inference cold start.

What happens to caches

Cache	At model switch
Hermes local system-prompt cache	Invalidated; rebuilt next turn; history is not deleted
Provider prefix cache	Not portable between models — cache entries are tied to the model and usually to the route/account
Old provider entries	May remain until expiration; they are not transferable tokens
Hermes history / state.db	Kept, unless explicitly compressed or deleted; re-sent next turn

Caveat verified in the current code: the Hermes cache_control policy does not give DeepSeek the standard explicit prompt cache over the OpenRouter chat-completions route — do not assume DeepSeek actually had prompt-cache hits without reading provider metrics. openrouter.response_cache is a different feature and must not be confused with prompt caching.

Cases not to confuse

Action	History	System prompt	Old-model cache
Normal turn, same route	kept	reused	possible hit per TTL/route
/model to another model	kept	rebuilt next turn	not reusable by the new model
Restart then session resume	restored from state.db	restored if compatible, else rebuilt	provider- and TTL-dependent
New session	absent unless passed explicitly	built	new prefix
Compression	middle summarized/rewritten	rebuild allowed	cache of the modified segment invalidated

Operational rule

A model change is a potential cost boundary. For long, price-sensitive work, avoid switching models mid-context. When the switch is necessary, treat the next call as potentially uncached and verify real cost in provider/Hermes metrics instead of assuming a cache hit.