> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI compatibility

> RunInfra exposes an OpenAI-shaped HTTP API for verified deployment endpoints. Use supported OpenAI clients with a RunInfra base URL, API key, and deployed model ID.

RunInfra's HTTP API follows OpenAI's request and response shapes for the endpoints it supports. If your code is already written against the OpenAI SDK, you change the base URL and API key, then pass the model ID your RunInfra deployment serves.

<Note>
  Use the generated snippet in **Settings > API Keys** or the **Deploy** tab for the exact production base URL. New snippets default to `https://api.runinfra.ai/v1`. If your workspace has an API-domain alias, keep the generated value.
</Note>

## Two-line migration

<CodeGroup>
  ```python Python theme={"dark"}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.runinfra.ai/v1",   # <- change
      api_key="YOUR_RUNINFRA_API_KEY",           # <- change
  )

  response = client.chat.completions.create(
      model="your-model-id",
      messages=[{"role": "user", "content": "Hello"}],
  )
  ```

  ```javascript TypeScript theme={"dark"}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://api.runinfra.ai/v1",   // <- change
    apiKey: "YOUR_RUNINFRA_API_KEY",          // <- change
  });

  const response = await client.chat.completions.create({
    model: "your-model-id",
    messages: [{ role: "user", content: "Hello" }],
  });
  ```

  ```bash curl theme={"dark"}
  curl https://api.runinfra.ai/v1/chat/completions \
    -H "Authorization: Bearer YOUR_RUNINFRA_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"your-model-id","messages":[{"role":"user","content":"Hello"}]}'
  ```
</CodeGroup>

Streaming, tools, structured outputs, and async OpenAI clients use the same request and response shapes where the selected deployment supports those features. Retry behavior still belongs to your client configuration, do not blindly retry charge-bearing requests after a partial stream or binary response may have reached your app.

## Endpoints supported today

| OpenAI endpoint                 | Supported | Notes                                                                                     |
| ------------------------------- | --------- | ----------------------------------------------------------------------------------------- |
| `POST /v1/chat/completions`     | Yes       | Streaming, tools, `response_format`                                                       |
| `POST /v1/responses`            | Yes       | Chat-completions compatibility adapter for compatible LLM and vision-language deployments |
| `POST /v1/embeddings`           | Yes       | Single string or array input                                                              |
| `POST /v1/images/generations`   | Yes       | Image generation deployments return OpenAI-shaped image data                              |
| `POST /v1/audio/transcriptions` | Yes       | Multipart file upload, Whisper-class models                                               |
| `POST /v1/audio/speech`         | Yes       | Text-to-speech, XTTS / Bark / Qwen3-TTS                                                   |
| `GET /v1/models`                | Yes       | Lists verified deployed models in your workspace                                          |

Anything not in this table is not currently supported. That includes `/v1/completions` (legacy, non-chat), `/v1/files`, `/v1/assistants`, `/v1/threads`, `/v1/batches`.

## What to change in your code

Three lines at most:

<Steps>
  <Step title="Change the base URL">
    `https://api.openai.com/v1` to the RunInfra base URL shown in your dashboard snippet, usually `https://api.runinfra.ai/v1`.
  </Step>

  <Step title="Change the API key">
    Use your RunInfra API key from [Settings > API Keys](https://runinfra.ai/settings/api-keys) instead of an OpenAI `sk-...` key.
  </Step>

  <Step title="Change the model id">
    OpenAI model names (e.g. `gpt-4o`) won't resolve on RunInfra. Pass the model id your RunInfra pipeline serves. `GET /v1/models` returns the list.
  </Step>
</Steps>

## Response parity

For the supported endpoints, RunInfra returns the same JSON shapes OpenAI does:

* Chat completions return `id`, `object`, `created`, `model`, `choices[]`, `usage`.
* Responses return OpenAI-shaped response events and non-streaming JSON for the supported adapter fields.
* Streaming deltas match OpenAI's SSE format, terminated by `data: [DONE]`.
* Image generation returns OpenAI-shaped image data for verified image deployments.
* Tool calls return `tool_calls` on the assistant message with `function.name` / `function.arguments`.
* Structured output accepts `response_format: { type: "json_object" }` and `response_format: { type: "json_schema", json_schema: {...} }`.
* Usage metadata follows the endpoint shape: token usage for LLMs, embedding counts for vector calls, and modality-native units for image and audio routes.

## Verified libraries and clients

These clients are verified with RunInfra's supported OpenAI-shaped endpoints:

<Columns cols={3}>
  <Card title="OpenAI SDKs" icon="code">
    `openai` on Python and Node. The exact SDK OpenAI publishes.
  </Card>

  <Card title="RunInfra SDK" icon="shield-check" href="/tools-sdks/runinfra-sdk">
    Native RunInfra helpers for scoped keys, pipeline IDs, typed errors, request IDs, audio, images, and webhook verification.
  </Card>

  <Card title="LangChain" icon="link" href="/integrations/langchain">
    `ChatOpenAI(openai_api_base=..., openai_api_key=...)`
  </Card>

  <Card title="LlamaIndex" icon="book-open" href="/integrations/llamaindex">
    `OpenAI(api_base=..., api_key=...)`
  </Card>

  <Card title="Vercel AI SDK" icon="zap" href="/integrations/vercel-ai-sdk">
    `createOpenAICompatible({ baseURL, apiKey })`
  </Card>

  <Card title="Instructor" icon="code">
    Use it through the OpenAI Python client for supported structured-output chat flows.
  </Card>

  <Card title="curl / fetch" icon="terminal">
    Plain HTTP works too. No SDK required.
  </Card>
</Columns>

## Known differences

<Warning>
  Only the endpoints in the table above are available. Features that depend on `/v1/assistants`, `/v1/threads`, or `/v1/batches` will not work until those surfaces ship.
</Warning>

* OpenAI model names (`gpt-4o`, `gpt-4.1-mini`) don't alias to RunInfra models. Pass the model id your deployment serves.
* `/v1/responses` is a chat-completions compatibility adapter. It does not implement state, include, reasoning, hosted tools, conversation-item, or background-job semantics.
* Model-specific sampling parameters (`logit_bias`, `logprobs`, `seed` determinism) depend on the serving backend behind your deployment.

## Unsupported parameters

The following OpenAI parameters are either ignored or rejected when passed to RunInfra:

| Parameter                                         | Behavior                                                     | Why                                                                                           |
| ------------------------------------------------- | ------------------------------------------------------------ | --------------------------------------------------------------------------------------------- |
| `service_tier`                                    | Ignored                                                      | RunInfra has its own deployment-mode model (Flex / Active); no service tier concept           |
| `store`                                           | Ignored                                                      | RunInfra does not persist completions for replay; use the request id header to correlate logs |
| `metadata`                                        | Echoed back in the response unchanged                        | Surfaced in audit logs (Enterprise); has no semantic effect                                   |
| `parallel_tool_calls=false`                       | Honored by vLLM / SGLang backends; serving-backend dependent | Some backends always parallel-call by default                                                 |
| `prediction` (speculative decoding hint)          | Ignored                                                      | RunInfra picks the draft model itself; see [Speculation](/deployments/speculation)            |
| `audio` (output format on chat)                   | Rejected                                                     | Use `/v1/audio/speech` for TTS instead                                                        |
| `web_search`, `file_search`, `computer_use` tools | Rejected                                                     | These are OpenAI-hosted tools, not function calls; bring your own implementation              |

## Error code mapping

RunInfra returns OpenAI-shaped error envelopes (`{ error: { message, type, code } }`) and uses HTTP status codes consistently:

| HTTP | OpenAI `error.type`     | When it fires                                                                                           |
| ---- | ----------------------- | ------------------------------------------------------------------------------------------------------- |
| 400  | `invalid_request_error` | Schema mismatch, missing field, unsupported parameter                                                   |
| 401  | `authentication_error`  | Bad or missing API key                                                                                  |
| 403  | `permission_error`      | Key lacks access to the requested pipeline                                                              |
| 404  | `not_found_error`       | Model id does not exist or is not deployed                                                              |
| 422  | `invalid_request_error` | Input too long, malformed JSON schema, bad image bytes, or idempotent chat/Responses replay unavailable |
| 429  | `rate_limit_error`      | Per-key budget exceeded; see [Rate limits](/api-reference/rate-limits)                                  |
| 500  | `server_error`          | RunInfra internal error; retryable with exponential backoff                                             |
| 502  | `server_error`          | Upstream serving backend transient failure; retryable                                                   |
| 503  | `server_error`          | All replicas busy, queue full; respect `Retry-After`                                                    |

The `X-Request-Id` response header carries a UUID you can quote when filing a support ticket. Always include it.

## Fallback behavior

When an upstream serving backend (vLLM, SGLang, TRT-LLM) does not support a parameter the client sent:

* **Numeric out-of-range** (e.g. `top_p=1.5`): clamped to the legal range, logged in the backend response header `X-Param-Adjustments`.
* **Unsupported feature** (e.g. `logit_bias` on a backend that does not support it): the parameter may be ignored by the serving backend, and `X-Param-Adjustments` lists what was dropped when the gateway can detect it.
* **Unsupported model capability** (e.g. tool calling on a non-instruction-tuned model): returns 400 `invalid_request_error` with a message naming the missing capability.

## Next steps

<Columns cols={2}>
  <Card title="API reference" icon="square-terminal" href="/api-reference/introduction">
    Endpoint-by-endpoint parameters and response fields.
  </Card>

  <Card title="RunInfra SDK" icon="shield-check" href="/tools-sdks/runinfra-sdk">
    Native SDK setup for optimized deployment access.
  </Card>

  <Card title="Tool calling cookbook" icon="wrench" href="/cookbook/tool-calling">
    Function calling with OpenAI tool schemas.
  </Card>

  <Card title="Rate limits" icon="gauge" href="/api-reference/rate-limits">
    Per-key limits and the `Retry-After` header.
  </Card>

  <Card title="Integrations" icon="plug" href="/integrations/overview">
    Framework-specific setup.
  </Card>
</Columns>