Idempotent retries

Add an Idempotency-Key when you may retry a non-streaming chat completion after a lost response or network failure. The header is optional. Use a non-blank ASCII value no longer than 255 characters.

response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Write one sentence."}],
    max_tokens=64,
    extra_headers={"Idempotency-Key": "YOUR_STABLE_IDEMPOTENCY_KEY"},
)

const response = await client.chat.completions.create(
  {
    model: "deepseek-v4-flash",
    messages: [{ role: "user", content: "Write one sentence." }],
    max_tokens: 64,
  },
  { headers: { "Idempotency-Key": "YOUR_STABLE_IDEMPOTENCY_KEY" } },
);

curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: YOUR_STABLE_IDEMPOTENCY_KEY" \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Write one sentence."}],"max_tokens":64}'

Reuse rules

Reuse the same key only with the same HTTP method, route, and request body.

Result	Gateway behavior
The first request completed and its response fits the replay cache	Returns the stored response and sets `X-RunInfra-Idempotent-Replay: true`.
The first request is still running	Returns `409 idempotency_conflict` with `Retry-After: 1`. Retry with the same key.
The key was used with different request parameters	Returns `400 invalid_request_error`. Use a different key only for a new logical request.
The stored response is too large to replay	Returns `422 idempotency_replay_unavailable`. Do not retry with a new key because that can start new inference work.

Streaming chat completions bypass this replay cache. An Idempotency-Key does not make a streaming request replayable.

Replay protection for non-streaming chat is best effort. If the replay store is unavailable, the request proceeds without deduplication instead of failing.

Client request ids

X-Client-Request-Id is an optional correlation header. The gateway echoes a valid value in the response. It is not a substitute for Idempotency-Key on chat completions. If you send it, use a non-blank ASCII value no longer than 512 characters.

⌘I

Getting started

Endpoints

Reference

Idempotent retries

Reuse rules

Client request ids

​Reuse rules

​Client request ids

Reuse rules

Client request ids