> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming Model APIs responses

> Stream OpenAI-compatible chat completion deltas and request client-visible usage.

Set `stream: true` to receive a Server-Sent Events response from `POST /v1/chat/completions`.

## Stream a response

Set `stream_options.include_usage` to `true` when your client needs the usage-only chunk.

<CodeGroup>
  ```python Python theme={"dark"}
  import os
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.runinfra.ai/v1",
      api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
  )
  stream = client.chat.completions.create(
      model="deepseek-v4-flash",
      messages=[{"role": "user", "content": "Count from one to three."}],
      max_tokens=64,
      stream=True,
      stream_options={"include_usage": True},
  )

  for chunk in stream:
      if chunk.usage:
          print(f"\nusage: {chunk.usage}")
          continue
      print(chunk.choices[0].delta.content or "", end="", flush=True)
  ```

  ```typescript TypeScript theme={"dark"}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://api.runinfra.ai/v1",
    apiKey: process.env.RUNINFRA_GATEWAY_KEY,
  });
  const stream = await client.chat.completions.create({
    model: "deepseek-v4-flash",
    messages: [{ role: "user", content: "Count from one to three." }],
    max_tokens: 64,
    stream: true,
    stream_options: { include_usage: true },
  });

  for await (const chunk of stream) {
    if (chunk.usage) console.error("usage:", chunk.usage);
    process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  }
  ```

  ```bash cURL theme={"dark"}
  curl -N https://api.runinfra.ai/v1/chat/completions \
    -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Count from one to three."}],"max_tokens":64,"stream":true,"stream_options":{"include_usage":true}}'
  ```
</CodeGroup>

## Event shape

The response uses these headers:

```http theme={"dark"}
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive
```

Content arrives in OpenAI-compatible `data:` events. Read partial output from `choices[].delta`.

```text theme={"dark"}
data: {"id":"chatcmpl_example","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"One"}}]}

```

When client-visible usage is enabled, the usage event has an empty `choices` array:

```text theme={"dark"}
data: {"choices":[],"usage":{"prompt_tokens":12,"completion_tokens":4,"total_tokens":16}}

```

The gateway forwards `data: [DONE]` when the upstream stream sends it.

```text theme={"dark"}
data: [DONE]

```

## Is usage always included?

No. The gateway always requests usage from the hosted model so it can settle the request. It forwards the usage-only event to your client only when your original request includes:

```json theme={"dark"}
{
  "stream_options": {
    "include_usage": true
  }
}
```

If you omit that option, content chunks still stream, but the usage-only chunk is hidden from your client. If the upstream stream omits usage, the gateway does not synthesize a client-visible usage chunk.

<Note>
  Streaming requests do not use the chat idempotency cache. See [Idempotent retries](/docs/api-reference/idempotent-retries) before retrying a stream.
</Note>