> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Model APIs limits

> Context, output, concurrency, and token-rate limits for hosted Model APIs calls.

These limits apply to hosted Model APIs chat completions.

## Token limits

| Limit                  |          Value | Behavior                                                        |
| ---------------------- | -------------: | --------------------------------------------------------------- |
| Context window         | 131,072 tokens | Published context limit for hosted chat completions.            |
| Total generated output |  32,768 tokens | The cap covers all generated choices in one request.            |
| Generated choices      |              4 | Candidate-count fields above four are rejected before dispatch. |

If you omit both `max_tokens` and `max_completion_tokens`, the gateway assigns the available 32,768-token output budget across the requested generation count. With one generation, the default is 32,768. With two, the default per generation is 16,384.

If you set both token fields, the gateway uses the larger value when checking the total output budget.

## Concurrency

| Scope        | Current behavior                                                                                                                          |
| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| API key      | At most 2 concurrent hosted requests.                                                                                                     |
| Workspace    | `min(2, max(1, floor(model concurrency / 4)))`. DeepSeek V4 Flash is configured for 8 shared requests, so the current workspace cap is 2. |
| Hosted model | DeepSeek V4 Flash is configured for 8 concurrent requests across workspaces.                                                              |

Concurrency is counted per hosted model. A slot is released when a non-streaming request completes or when a streaming response settles. A rejected request does not keep the slot.

When a limit is reached, the gateway returns `429` with `Retry-After` and hosted-limit headers.

## Workspace tokens per minute

DeepSeek V4 Flash has a workspace limit of 200,000 tokens in a rolling 60-second window.

Before dispatch, the gateway reserves:

```text theme={"dark"}
estimated input tokens + maximum requested output tokens
```

The input estimate is at least one token and otherwise uses one token per four request-body characters. After a successful response, the gateway replaces the reservation estimate with the reported prompt and completion token counts.

This means a request can receive `429` before inference starts when its reservation would exceed the current workspace window. If one request is too large for the limit, reduce its output cap or request size before retrying.

## Admission error codes

| `error.code`                            | Limit                       |
| --------------------------------------- | --------------------------- |
| `hosted_per_key_concurrency_limit`      | API-key concurrency         |
| `hosted_workspace_concurrency_limit`    | Workspace concurrency       |
| `hosted_shared_model_concurrency_limit` | Shared model concurrency    |
| `hosted_workspace_tpm_limit`            | Workspace tokens per minute |

The response also includes:

```http theme={"dark"}
Retry-After: <seconds>
X-Hosted-Limit-Scope: <scope>
X-Hosted-Limit-Value: <limit>
X-Hosted-Limit-Retry-After-Seconds: <seconds>
```

The API key's requests-per-minute limit runs before hosted-model admission. It is a separate limit and uses the `X-RateLimit-*` response headers.

See [Rate limits](/docs/api-reference/rate-limits) for the exact `429` envelopes, messages, and caller actions.