What we store per request
One usage row per request, plus the billing transaction that settles it, holding only these fields between them.
That is the complete list. Both records do carry a free-form JSON column, so the honest statement is not “no column could hold text” but what the writer actually puts there: on the usage row, the request id, the credit held while the request runs, and a flag recording whether a promotional price applied; on the billing transaction, the settled token counts, the price snapshot, and the settlement bookkeeping. Nothing else is written on any code path, and the usage row’s value is asserted as an exact object in our test suite, so adding a key to it fails the build.
The replay cache, the one window you can open
Sending anIdempotency-Key holds that request’s response for up to 24 hours, so a retry with the same key returns the same answer instead of running, and charging for, the work twice. Send no key and nothing is written.
- It holds the response, not the request. The request body is turned into a SHA-256 fingerprint first, and the fingerprint is what is stored, so a key reused with different parameters can be rejected rather than answered with the wrong reply.
- It expires on its own. The entry carries its 24 hour expiry when it is written, rather than waiting for a cleanup job.
- Streamed responses are never stored. The cache refuses an event stream outright. A duplicate of a streamed request receives a receipt instead: the usage and the cost of the original, with the message content explicitly
null, because we did not keep the generated text and will not invent it. - Responses larger than 512 KB are not stored. Only the byte count is kept, and a duplicate is told the original is too large to replay.
- It is scoped to your workspace and your key. The cache key is derived from both, so no other tenant can reach an entry.
- It is purpose bound. The only thing that reads an entry is the code path answering a duplicate of the same request. It feeds no analytics, no training, and no support tooling.
- What it holds is the response we built, not the one the model returned. Every response is projected through an allowlist before it leaves us: only the fields OpenAI documents survive, plus our own
runinfranamespace. Serving software can echo your prompt back to us on its own response object, and that field is dropped by construction rather than by a rule someone remembered to write, so it is neither in your reply nor in the cache entry.
What we never store
Error reporting runs with request-body capture and stack-frame variable capture both switched off, so a server exception cannot carry request content into it as a side effect. Product analytics carries no user-written text at all, enforced by tests that plant a needle in a payload and assert it is absent from what would be transmitted.
What exists only while the request runs
Serving a request means holding it in memory for as long as the request takes. That is unavoidable, and it is the honest boundary of any inference provider’s retention claim. Your messages live in memory on the machine generating the answer and are released with the call. Repeated leading context is kept as a cached prefix in that machine’s working memory, never written to our database, and it is what makes cached input cheaper than fresh input. On every model where we publish a cached input price, that cache is partitioned per workspace, so a cached prefix is only ever reachable by the workspace that created it. The partition, the cached token count in the response, and the cached rate on the bill are switched on by one and the same setting, and all three additionally require the partitioning secret to be present, so a published cached rate is itself the signal that the isolation is active. On models with no cached rate, no cached input discount is billed and no cached token count is disclosed.What reaches the machine serving the model
Not your workspace id. Where a per-workspace value is needed to keep caches separate, we send a one-way HMAC-SHA256 digest of it instead. Being precise about what that buys you: the digest identifies no one to the serving host, and no one to anybody who does not hold our key. It is not anonymous to us. We hold the key, so we can recompute the digest for a workspace id we already know and match it back. That is deliberate, because it is what lets us debug a cache partition, and it is why we call the value unlinkable by the recipient rather than unlinkable full stop.What a rejected or failed request leaves behind
A rejected request writes no text from your request. When a request is refused as invalid with400 or 422, what reaches our operational log is the request id, the status, the error type, and the error code. The human-readable message goes back to you in the response and is not logged.
A failed request logs our own error, not yours. On a 5xx we log the error object our code or the upstream produced: its class name, its message truncated to 500 characters, and its stack truncated to 2,000 characters, all passed through a credential redactor first. That is diagnostic text we or the serving software wrote. No code path puts your messages into it. We will not claim more than that, because an upstream error string is not ours to guarantee the wording of.
What this is not
- This describes how the service behaves today, not a contractual zero-retention guarantee. If you need retention terms in writing for a procurement or compliance review, contact us and we will handle it as a contract rather than a docs page.
- We do not claim “zero data retention”. Billing metadata is retained, and the replay window retains a response body when you open it. Any provider that tells you it keeps literally nothing while still charging you per token is describing something that cannot work.
- Your own copies are yours to manage. If you log requests and responses on your side, that retention is governed by your systems.
Related
Idempotent retries
The header that opens the 24 hour window, and what it replays.
Authentication
How keys are stored, rotated, and retired.