Session affinity

Send an x-session-id header with a stable value of your choosing, and every request carrying that value is kept together as one session. On models with prefix caching, this helps keep the session’s cache warm across turns, which can lower time to first token and cached input cost on multi-turn conversations and fan-out agent workloads. The header is optional, it works on streaming and non-streaming requests alike, and there is nothing to create or clean up. A session exists by being used. The same header applies to /v1/messages. Send x-session-id there too: metadata.user_id reaches the model as user and is not a routing identity on its own.

response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Summarize the thread so far."}],
    max_tokens=16384,
    extra_headers={"x-session-id": "chat-7c2f1a"},
)

const response = await client.chat.completions.create(
  {
    model: "deepseek-v4-flash",
    messages: [{ role: "user", content: "Summarize the thread so far." }],
    max_tokens: 16384,
  },
  { headers: { "x-session-id": "chat-7c2f1a" } },
);

curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "x-session-id: chat-7c2f1a" \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Summarize the thread so far."}],"max_tokens":16384}'

The identity headers

Header	What it names
`x-session-id`	This request’s own session. Requests sharing a value share the session’s warm cache.
`x-parent-session-id`	The parent session this request should co-locate with. Read when `x-session-id` is absent, so a fan-out worker reuses its parent’s warm prefix.
`x-session-affinity`	The earlier spelling of `x-session-id`. Still honored. Prefer `x-session-id` in new code.
`x-runinfra-session`, or `runinfra.sessionId` in the request body	The same session identity carried as a request field or its own header. When present and valid, it is read ahead of the three headers above.

One identity decides routing per request, chosen in this order: a valid runinfra.sessionId body field, then the x-runinfra-session header, then x-session-id, x-parent-session-id, x-session-affinity, then the body field prompt_cache_key. With none of them, the request may be grouped by its opening messages, described below; otherwise it keeps workspace-level grouping. An identity sharpens that grouping from the workspace to the session.

Spec	Value
Value length, `x-session-id` and its two siblings	1 to 256 characters, after surrounding whitespace is trimmed
Value outside that range	Ignored. The other session headers are not consulted; `prompt_cache_key` and the placement rules below still apply
Value shape, `runinfra.sessionId` and `x-runinfra-session`	1 to 128 characters from letters, digits, `.`, `_`, `:` and `-`. An invalid value is ignored and the session headers are read instead
Comparison	Exact and case sensitive
No identity sent	May be grouped by the opening messages, otherwise workspace-level grouping

Fan-out: workers follow the parent

An agent that fans out sends the orchestrator’s turns on its own session, and each worker names that session as its parent. The workers then read the same warm prefix the orchestrator built, the shared system prompt and plan, with no coordination between them.

# Orchestrator turn, on its own session
curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "x-session-id: run-4127" \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Plan the review, then dispatch workers."}],"max_tokens":16384}'

# Worker request, co-located with the parent
curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "x-parent-session-id: run-4127" \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Review module A against the shared plan."}],"max_tokens":16384}'

x-session-id wins when both headers are present, so a worker that should follow its parent sends x-parent-session-id only. Give a worker its own x-session-id once it carries its own multi-turn context worth keeping warm.

`prompt_cache_key`, `user`, and the opening messages

Clients that already send the standard prompt_cache_key request field get the same routing with no new headers: when no session identity is present, prompt_cache_key is read as the identity. The field is consumed at the gateway. It shapes routing and is removed before the request reaches the model. prompt_cache_options and prompt_cache_retention are rejected with 400 hosted_parameter_not_supported. user is not a routing identity. It remains a standard request field and reaches the model unchanged. Without a session identity or prompt_cache_key, the gateway may place the request by its opening messages instead: the user value together with the leading system or developer text and the first user message. Two requests that share a user value but open with different messages may be placed apart. A request with no message text keeps workspace-level placement. Prefer prompt_cache_key, or a session header, in new code. A session header keeps an established session’s requests close to its warm cache even as load changes, until the session has been idle for 15 minutes or its warm cache becomes unavailable. A body hint such as prompt_cache_key groups requests by value but does not hold an established session in place. Neither controls cache retention. prompt_cache_options and prompt_cache_retention remain unsupported: each returns 400 hosted_parameter_not_supported.

How placement behaves

A new session’s placement considers current load. An established session keeps its placement while it stays active; load changes alone never move it.
A session’s placement is remembered for 15 minutes of idle time, the window GET /v1/models publishes as session_affinity_idle_ttl_seconds. After a longer gap, the next request is placed afresh, with load considered again. This is a routing window, not a cache retention promise; what the cache kept is covered below.
Affinity is a preference, not a delivery constraint. If the session’s warm cache is unavailable, the request is still served and the session settles where it was served, so its cache rebuilds from that request onward.
Identity values are scoped to your workspace, and the raw value is never stored. The same literal value sent by another workspace shares nothing with yours.

Response headers

These response headers are not yet live; they will be announced when they ship. Once live, a non-streaming /v1/chat/completions response to a request that named a session will report what the prefix cache did for it, on every hosted model whose per-workspace cache isolation is active, the state GET /v1/models reports as cache_isolation: "isolated". Until then, read usage.prompt_tokens_details.cached_tokens in the response body, which is live today.

x-session-state: <cold | warm | restored>
x-session-tier: <gpu | ram | nvme>
x-session-cached-share: <two-decimal share>

Header	Meaning
`x-session-cached-share`	The share of this request’s input tokens that were served from cache, to two decimal places, and `0.00` for a request with no input tokens.
`x-session-state`	`cold`, `warm` or `restored`. Not sent yet; this header ships with the session reporting described above.
`x-session-tier`	`gpu`, `ram` or `nvme`. Not sent yet; this header ships with the session reporting described above. The header’s `ram` is the tier `GET /v1/models` lists as `host_ram`.

Read usage.prompt_tokens_details.cached_tokens and the cached input rate on the bill today. When the session headers ship, x-session-cached-share will report the share too. All three headers are absent, and the response is otherwise unchanged, when any of the following holds:

The model’s per-workspace cache isolation is not active: cache_isolation reads shared on GET /v1/models.
The request was routed by prompt_cache_key, or kept workspace-level placement. A prompt_cache_key shapes routing but opens no session, so there is nothing to report. The headers appear when the request named a session, and when it was placed by its opening messages.
The response is streamed. Streaming responses carry no session headers.
The call went through /v1/messages, which forwards no session headers. They appear only on /v1/chat/completions responses.

Affinity keeps the session warm, the tiers keep the prefix

Affinity and the session cache tiers are two halves of one mechanism. Affinity keeps a session’s requests close to the caches that already hold its prefix, and the tiers keep that prefix alive between requests. Keep windows exist on one model today, GLM 5.3 Flash (glm-5-3-flash). There, the prefix behind a session you name is kept across idle gaps in storage tiers with different keep windows, the tier values GET /v1/models lists under cache_tiers:

Tier	Idle window	On the session’s next request
`gpu`	Up to 6 hours since the last request	Reused immediately
`ram`	Up to 6 hours since the last request	Brought back for the request
`nvme`	Up to 72 hours since the last request	Brought back for the request

Those windows are ceilings, not guarantees: idle state can be dropped earlier under capacity pressure. What survived a pause shows in the cached-token count of the first turn after it and, once the session headers ship, in that response’s x-session-cached-share, whichever tier it came back from. When a window ends the state is gone, and the next request with that identity starts cold. Session storage across those windows comes at no charge. On every other model these keep windows do not exist: the cache is the ordinary kind with best-effort retention, evicted under capacity pressure rather than kept for a window. GET /v1/models states each model’s cache_isolation, cache_tiers and cache_retention; the keep windows above are the idle limits for named sessions on this model, published here and on Data retention.

Chat completions

The request contract these headers ride on.

Idempotent retries

Retry a call without duplicate work or a second charge.

Prompt caching

What keeps a prefix reusable across an agent’s turns.

Data retention

What stays with the model between requests, and for how long.

Start here

Guides

API reference

The identity headers

Fan-out: workers follow the parent

`prompt_cache_key`, `user`, and the opening messages

How placement behaves

Response headers

Affinity keeps the session warm, the tiers keep the prefix

Chat completions

Idempotent retries

Prompt caching

Data retention

​The identity headers

​Fan-out: workers follow the parent

​prompt_cache_key, user, and the opening messages

​How placement behaves

​Response headers

​Affinity keeps the session warm, the tiers keep the prefix

​Related

Chat completions

Idempotent retries

Prompt caching

Data retention

The identity headers

Fan-out: workers follow the parent

`prompt_cache_key`, `user`, and the opening messages

How placement behaves

Response headers

Affinity keeps the session warm, the tiers keep the prefix

Related