Idempotent retries

Send an Idempotency-Key header when a lost response or a network failure might make you retry. It is optional, it works on streaming and non-streaming requests alike, and it guarantees one generation and one charge. The same header and replay rules apply to /v1/messages. Use a non-blank ASCII value of 255 characters or fewer. One fresh key per logical request, reused only when you are retrying that same request. A UUID generated at the call site is a convenient source.

response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Write one sentence."}],
    max_tokens=64,
    extra_headers={"Idempotency-Key": "YOUR_STABLE_IDEMPOTENCY_KEY"},
)

const response = await client.chat.completions.create(
  {
    model: "deepseek-v4-flash",
    messages: [{ role: "user", content: "Write one sentence." }],
    max_tokens: 64,
  },
  { headers: { "Idempotency-Key": "YOUR_STABLE_IDEMPOTENCY_KEY" } },
);

curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: YOUR_STABLE_IDEMPOTENCY_KEY" \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Write one sentence."}],"max_tokens":64}'

What a reused key does

Reuse the same key only with the same HTTP method, route, and request body. A key sent with a different body is a different logical request, and it is refused with 400 idempotency_mismatch rather than answered with the wrong reply. If you would rather opt out entirely, omit the header: a request without an Idempotency-Key runs with no deduplication at all.

Retrying a stream

A stream is not replayed byte for byte. We do not store the tokens we deliver, so there is nothing to send again, and a live event stream is never stored for replay. What the key protects on a streaming request is the part that costs money.

Situation	What you get
The original stream is still open	`409 idempotency_conflict` with `Retry-After: 1`. No second generation starts.
The original stream finished and settled	`200` with `X-RunInfra-Idempotent-Replay: true` and a non-streaming JSON receipt. Nothing is regenerated and nothing is billed again.
The original delivered nothing and was billed nothing	The key is released, so a retry runs normally, exactly as a request without a key would.

The receipt is an ordinary chat completion envelope, so an OpenAI-compatible client can parse it. content and finish_reason are null because the delivered text was never stored, and the replay facts ride in idempotent_replay.

{
  "id": "chatcmpl-8f0a1c2e-...",
  "object": "chat.completion",
  "created": 1786800000,
  "model": "qwen3-8-27b",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": null },
      "finish_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 1000,
    "completion_tokens": 500,
    "total_tokens": 1500,
    "prompt_tokens_details": { "cached_tokens": 200 },
    "cost": 0.00024,
    "runinfra": { "cost_microcents": 24000, "cached_input_tokens": 200 }
  },
  "idempotent_replay": {
    "replayed": true,
    "streamed": true,
    "content_replayed": false,
    "regenerated": false,
    "billed_again": false,
    "original_status": 200,
    "stream_outcome": "completed",
    "usage_basis": "provider_reported",
    "cost_microcents": 24000
  }
}

X-RunInfra-Idempotent-Replay is the signal on every route: read it before handing a response to a streaming parser. The idempotent_replay object is returned by /v1/chat/completions; on /v1/responses the reply is converted to a Responses object and only the header survives. If you need the generated text after a dropped stream, send a new request with a new key. That is new paid inference. Reusing the old key returns the receipt, not the text.

Only one charge per key

A retried stream is charged once: a duplicate settlement for the same key is refused rather than billed twice. On chat completions, responses, and messages (streamed or not) this is best effort and fails open rather than failing your request: if a key cannot be checked, the request proceeds without deduplication instead of returning an error, which means a retry can run and settle on its own. We would rather serve a chargeable chat request than fail it. Keyed embeddings, rerank, and transcription requests instead return 503 idempotency_unavailable with Retry-After; retry them with the same key.

Client request ids

X-Client-Request-Id is an optional correlation header. The gateway echoes a valid value back on the response. It is not a substitute for Idempotency-Key. Use a non-blank ASCII value of 512 characters or fewer.

Streaming

The frame shapes a retried stream is not replayed as.

Data retention

The 24 hour replay window is the one place a response body is stored.

Start here

Guides

API reference

Idempotent retries

What a reused key does

Retrying a stream

Only one charge per key

Client request ids

Streaming

Data retention

​What a reused key does

​Retrying a stream

​Only one charge per key

​Client request ids

​Related

Streaming

Data retention

What a reused key does

Retrying a stream

Only one charge per key

Client request ids

Related