Idempotency-Key, and audio uploaded through the large-file lane for up to 2 hours after the upload. On glm-5-3-flash, a session’s model working state (not a copy of your text) is kept only on the serving side for up to 72 hours after your last request.
What we store per request
One usage record per request, plus the billing transaction that settles it. Between them they hold the fields below and settlement metadata: where the token counts came from, how a stream ended, interim token counts, and the settlement timestamp. None of these values contains your content.
Both records also carry free-form metadata. It holds identifiers, billing bookkeeping (token counts, the price snapshot, settlement state), and machine-readable outcome codes. No prompt, completion, or header is written to these records. The one text-shaped value is a redacted, length-capped copy of the serving side’s own rejection sentence when a request is refused as invalid; it describes the refusal, never your content. The groups below describe these values.
The pseudonymous identifier cannot be reversed to recover anything you sent, but we still treat it as pseudonymous data, never as anonymous.
The billing transaction carries the same settled counts, price snapshot, status, and pseudonymous identifier. Neither record receives your messages, your files, or the model’s output.
The replay window
Sending anIdempotency-Key holds that request’s response for up to 24 hours, so a retry with the same key returns the same answer instead of running, and charging for, the work twice. Send no key and no response body is held.
- It holds the response, not the request. The request body itself is not kept; only a fingerprint of it that cannot be turned back into the body is, so a key reused with different parameters can be rejected rather than answered with the wrong reply.
- It expires on its own. An entry is gone 24 hours after it is written.
- Streamed responses are never stored. The cache refuses an event stream outright. A duplicate of a streamed request receives a receipt instead: the usage and the cost of the original, with the message content explicitly
null, because we did not keep the generated text and will not invent it. - Responses larger than 6 MiB are not stored. Only the byte count is kept, and a duplicate is told the original is too large to replay.
- It is scoped to your workspace and your key. An entry is only ever visible to the workspace and key that wrote it, so no other tenant can reach it.
- It is purpose bound. The only thing that reads an entry is the code path answering a duplicate of the same request. It feeds no analytics, no training, and no support tooling.
- What it holds is the response we built, not the one the model returned. Every response is projected through an allowlist before it leaves us: only the documented chat completion response fields survive, plus our own
runinfranamespace. Any field beyond that allowlist, including any echo of your prompt that arrives with the model output, is dropped by construction rather than by a rule someone remembered to write, so it is neither in your reply nor in the cache entry.
The audio upload window
Audio above 4 MiB goes through the large-file lane: you mint a one-shot upload URL withPOST /v1/audio/uploads, PUT the bytes to it, then transcribe with upload_id in place of file. Those are three separate requests, so the uploaded audio has to outlive the request that carried it. It is held for up to 2 hours after the upload, whatever the audio’s length. A successful transcription deletes it at once, because an id is one-shot. An upload that is never transcribed, or whose transcription fails, is deleted automatically when its 2 hour window ends. Send the audio as a direct file part and none of this applies: the bytes exist only for the request that carries them.
What we never store
Audio uploaded through the large-file lane is the one input with a window of its own, described under The audio upload window.
Error reporting is configured so that a server exception cannot carry request bodies or local variables with it, so request content cannot leak into it as a side effect. Product analytics carries no user-written text at all.
What exists only while the request runs
Serving a request means holding it in memory for as long as the request takes. That is unavoidable, and it is the boundary of any inference provider’s retention claim. Your messages exist only on the serving side for as long as the call takes and are released with it. Repeated leading context is reused as a cached prefix, held only on the serving side and never written to our database, and it is what makes cached input cheaper than fresh input. On every model where we publish a cached input price, cached prefixes in thegpu tier are isolated per workspace, so a prefix in that tier is only ever reused by the workspace that created it. The cached token count and the cached rate are enabled only while that isolation is active. A published cached rate signals this isolation; any additional cache tier’s scope is stated under Cached working state. On models with no cached rate, no cached input discount is billed and no cache hit is disclosed; the billed cached-token count is 0.
Cached working state
The serving side does not receive your workspace id. Where it needs a per-workspace value to keep cached prefixes separate, it receives a value derived from your workspace id that cannot be reversed to it. Being precise about what that buys you: the value identifies no one to the serving side, and no one to anybody outside RunInfra. It is not anonymous to us. We can reproduce it for a workspace id we already know and match it back. That is deliberate, because it is what lets us investigate a cache isolation question, and it is why we call the value unlinkable by the recipient rather than unlinkable full stop. What remains between requests is bounded, and one model carries keep windows. On GLM 5.3 Flash (glm-5-3-flash), the model’s working memory of your prompt is kept between requests so a returning session can reuse it. That working memory is the key-value cache behind the cached prefix described above. It is held only on the serving side for up to 72 hours after your last request.
- It is the model’s working state, not a stored copy of your text. What is kept is the numeric state the model attends over while generating, not a transcript of your messages.
- It stays on the serving side. Cached working state is never stored in our database. The session metadata described above does not contain that state.
- It is scoped to your workspace. A session’s state is only ever restored for the workspace that created it. Another workspace sending the same bytes is scoped separately and finds nothing of yours.
- It is deleted when its keep window ends or capacity pressure requires it, whichever comes first. At 72 hours from your last request the state expires. Idle state can be dropped earlier. The windows above are ceilings on how long the state can exist, not guarantees that it will.
- There is no charge for it. Keeping the state adds nothing to your bill.
deepseek-v4-pro) keeps its prefix cache across its gpu and host_ram tiers between requests, evicted under capacity pressure, with no keep window. Per-workspace isolation on this model covers the gpu tier; the host_ram tier that extends it is not isolated per workspace today.
On every other model with a published cached input rate, including Qwen3.8 Flash Next (qwen3-8-flash-next), these keep windows do not exist. Their listing reports cache_isolation as isolated; the cached prefix uses the published gpu tier and is evicted under capacity pressure rather than kept for a window. A model with no cached rate is treated as shared: cache hits are not disclosed or discounted, and the billed cached-token count is 0.
GET /v1/models states each chat model’s behavior as cache_isolation, cache_tiers, and cache_retention; cache_isolation reads shared whenever no cached rate is published. cache_tiers and cache_retention carry what has been proven for each model’s prefix cache; deepseek-v4-pro, for example, reports the gpu and host_ram tiers. The session keep windows above are idle limits and are published here, not in that listing.
Rejected and failed requests
A rejected request writes no text from your request. When a request is refused as invalid with400 or 422, what reaches our operational log is the request id, the status, the error type, the error code, your workspace id, and the model id you asked for. When the refusal is raised after the request was dispatched, because a message or image could not be accepted or the model rejected the request, the message you receive is logged with them, after a credential redactor and a length cap. That message is wording our own code produces, not a copy of your request.
On a 5xx we log the error object our code or the upstream produced, passed through a credential redactor first. That is diagnostic text about the failure, not your request: no code path puts your messages into it.
Scope
- Retention terms in writing, for a procurement or compliance review, are a contract conversation. Contact us.
- The honest label is bounded retention, not zero: billing metadata is kept, the replay window keeps a response body when you open it, and uploaded audio and cached session working state have the windows described above.
- Your own copies are yours to manage. If you log requests and responses on your side, that retention is governed by your systems.
Related
Idempotent retries
The header that opens the 24 hour window, and what it replays.
Authentication
How keys are stored, rotated, and retired.