Skip to main content
These limits apply to hosted Model APIs chat completions.

​
Token limits

If you omit both max_tokens and max_completion_tokens, the gateway assigns the available 32,768-token output budget across the requested generation count. With one generation, the default is 32,768. With two, the default per generation is 16,384. If you set both token fields, the gateway uses the larger value when checking the total output budget.

​
Concurrency

Concurrency is counted per hosted model. A slot is released when a non-streaming request completes or when a streaming response settles. A rejected request does not keep the slot. When a limit is reached, the gateway returns 429 with Retry-After and hosted-limit headers.

​
Workspace tokens per minute

DeepSeek V4 Flash has a workspace limit of 200,000 tokens in a rolling 60-second window. Before dispatch, the gateway reserves:
The input estimate is at least one token and otherwise uses one token per four request-body characters. After a successful response, the gateway replaces the reservation estimate with the reported prompt and completion token counts. This means a request can receive 429 before inference starts when its reservation would exceed the current workspace window. If one request is too large for the limit, reduce its output cap or request size before retrying.

​
Admission error codes

The response also includes:
The API key’s requests-per-minute limit runs before hosted-model admission. It is a separate limit and uses the X-RateLimit-* response headers. See Rate limits for the exact 429 envelopes, messages, and caller actions.