Session affinity

Send an x-session-id header with a stable value of your choosing, and every request carrying that value is routed to the same serving replica. On models with prefix caching, the replica’s cache stays warm across turns, which lowers time to first token and cached input cost on multi-turn conversations and fan-out agent workloads. The header is optional, it works on streaming and non-streaming requests alike, and there is nothing to create or clean up. A session exists by being used.

response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Summarize the thread so far."}],
    max_tokens=16384,
    extra_headers={"x-session-id": "chat-7c2f1a"},
)

const response = await client.chat.completions.create(
  {
    model: "deepseek-v4-flash",
    messages: [{ role: "user", content: "Summarize the thread so far." }],
    max_tokens: 16384,
  },
  { headers: { "x-session-id": "chat-7c2f1a" } },
);

curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "x-session-id: chat-7c2f1a" \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Summarize the thread so far."}],"max_tokens":16384}'

The identity headers

Header	What it names
`x-session-id`	This request’s own session. Requests sharing a value share a replica.
`x-parent-session-id`	The parent session this request should co-locate with. Read when `x-session-id` is absent, so a fan-out worker lands on its parent’s replica.
`x-session-affinity`	The earlier spelling of `x-session-id`. Still honored. Prefer `x-session-id` in new code.

One identity decides routing per request, chosen in this order: x-session-id, then x-parent-session-id, then x-session-affinity, then the body fields prompt_cache_key and user. With none of them, the request keeps workspace-level placement: your workspace’s requests still group onto a replica rather than scattering, and an identity sharpens that grouping from the workspace to the session.

Spec	Value
Value length	1 to 256 characters, after surrounding whitespace is trimmed
Value outside that range	Ignored. The request proceeds as if no session identity had been sent; the next header in the order is not consulted
Comparison	Exact and case sensitive
No identity sent	Workspace-level placement

Fan-out: workers follow the parent

An agent that fans out sends the orchestrator’s turns on its own session, and each worker names that session as its parent. The workers then read the same warm prefix the orchestrator built, the shared system prompt and plan, with no coordination between them.

# Orchestrator turn, on its own session
curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "x-session-id: run-4127" \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Plan the review, then dispatch workers."}],"max_tokens":16384}'

# Worker request, co-located with the parent
curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "x-parent-session-id: run-4127" \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Review module A against the shared plan."}],"max_tokens":16384}'

x-session-id wins when both headers are present, so a worker that should follow its parent sends x-parent-session-id only. Give a worker its own x-session-id once it carries its own multi-turn context worth keeping warm.

`prompt_cache_key` and `user`

Clients that already send the standard prompt_cache_key request field get the same routing with no new headers: when no session header supplies an identity, prompt_cache_key is read as the identity. The field is consumed at the gateway. It shapes routing and is removed before the request reaches the model. user is the older spelling of the same hint and is read when prompt_cache_key supplies none. It remains a standard request field and still reaches the model unchanged. Prefer prompt_cache_key, or a session header, in new code. Two differences from the headers are worth knowing. A session header pins an established session’s placement, so it stays put as conditions change, while a body hint routes deterministically by value but does not pin. And prompt_cache_options and prompt_cache_retention remain unsupported: each returns 400 hosted_parameter_not_supported.

How placement behaves

A new session is placed with current load considered, and an established session then keeps its replica. Load changes alone never move it.
Placement is a routing preference, not a delivery constraint. If a session’s replica is unavailable, the request fails over to the next replica in a deterministic order.
Identity values are scoped to your workspace, and stored routing records carry only a hash, never the raw value. The same literal value sent by another workspace shares nothing with yours.

Chat completions

The request contract these headers ride on.

Idempotent retries

Retry a call without duplicate work or a second charge.

Start here

Guides

API reference

The identity headers

Fan-out: workers follow the parent

`prompt_cache_key` and `user`

How placement behaves

Chat completions

Idempotent retries

​The identity headers

​Fan-out: workers follow the parent

​prompt_cache_key and user

​How placement behaves

​Related

Chat completions

Idempotent retries

The identity headers

Fan-out: workers follow the parent

`prompt_cache_key` and `user`

How placement behaves

Related