> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# API reference

> OpenAI-compatible inference API. Set base_url once, reach verified deployments with a workspace-scoped key or a pipeline-scoped key.

RunInfra exposes an **OpenAI-compatible HTTP API** for verified deployments. Point the OpenAI Python or JavaScript SDK, the native RunInfra SDK, or documented integrations such as LangChain, LlamaIndex, and the Vercel AI SDK at the dashboard-generated base URL with a RunInfra API key.

<Columns cols={2}>
  <Card title="Base URL" icon="link" horizontal>
    `https://api.runinfra.ai/v1`
  </Card>

  <Card title="Auth scheme" icon="key" horizontal>
    `Authorization: Bearer YOUR_RUNINFRA_API_KEY`
  </Card>
</Columns>

## Key scopes

RunInfra supports **two types of API keys**, serving different integration shapes. Most customers should use workspace-scoped keys.

<Columns cols={2}>
  <Card title="Workspace-scoped (recommended)" icon="layers" horizontal>
    One key reaches verified deployed models in your workspace. The `model` field in the request body selects the target through the OpenAI SDK base URL pattern.
  </Card>

  <Card title="Pipeline-scoped" icon="route" horizontal>
    One key is bound to a single optimized pipeline. The pipeline ID sits in the URL path: `/v1/{pipelineId}/chat/completions`.
  </Card>
</Columns>

Create either at [Settings > API Keys](https://runinfra.ai/settings/api-keys).

## Supported endpoints

<Columns cols={2}>
  <Card title="POST /v1/chat/completions" icon="messages-square">
    Chat completions with streaming, tools, and structured output.
  </Card>

  <Card title="POST /v1/responses" icon="message-circle-code">
    Responses-shaped adapter over compatible chat-completions deployments.
  </Card>

  <Card title="POST /v1/embeddings" icon="braces">
    Vector embeddings for semantic search and RAG.
  </Card>

  <Card title="POST /v1/rerank" icon="list-filter">
    Text reranking for TEI deployments and multimodal document reranking for compatible vLLM vision rerank deployments.
  </Card>

  <Card title="POST /v1/images/generations" icon="image">
    Image generation from verified image deployments.
  </Card>

  <Card title="POST /v1/audio/speech" icon="volume-2">
    Text-to-speech. Binary audio response.
  </Card>

  <Card title="POST /v1/audio/transcriptions" icon="mic">
    Speech-to-text. Multipart audio upload.
  </Card>

  <Card title="GET /v1/models" icon="list">
    List verified deployed models in your workspace.
  </Card>
</Columns>

## Drop-in usage

<RequestExample>
  ```python Python theme={"dark"}
  from openai import OpenAI

  # Workspace-scoped key reaches any verified model in your workspace.
  client = OpenAI(
      base_url="https://api.runinfra.ai/v1",
      api_key="YOUR_RUNINFRA_API_KEY",
  )

  # List models
  for model in client.models.list().data:
      print(model.id)

  # Chat completion
  response = client.chat.completions.create(
      model="llama-3.3-70b",
      messages=[{"role": "user", "content": "Hello"}],
  )
  print(response.choices[0].message.content)
  ```

  ```javascript JavaScript theme={"dark"}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://api.runinfra.ai/v1",
    apiKey: "YOUR_RUNINFRA_API_KEY",
  });

  // List models
  const models = await client.models.list();
  console.log(models.data.map((m) => m.id));

  // Chat completion
  const response = await client.chat.completions.create({
    model: "llama-3.3-70b",
    messages: [{ role: "user", content: "Hello" }],
  });
  console.log(response.choices[0].message.content);
  ```

  ```bash cURL theme={"dark"}
  curl https://api.runinfra.ai/v1/chat/completions \
    -H "Authorization: Bearer YOUR_RUNINFRA_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "llama-3.3-70b",
      "messages": [{"role": "user", "content": "Hello"}]
    }'
  ```
</RequestExample>

<Note>
  If your dashboard snippet shows a different production base URL, keep the generated value. Workspace-scoped keys use `/v1`; pipeline-scoped snippets may include `/v1/{pipelineId}`.
</Note>

## Authentication

<ParamField header="Authorization" type="string" required>
  Bearer token. Keys start with `rp_live_` and are hashed at rest with SHA-256. Rotation and expiration are built in. See [Key management](/api-reference/authentication).
</ParamField>

<Warning>
  Never embed API keys in client-side code. They grant access to your workspace's inference budget. Use a backend proxy for browser-originated requests, or a Vercel Edge Function / Cloudflare Worker as a thin gateway.
</Warning>

## Rate limits by plan

Every API key carries a per-minute request budget. The default budget is set by your workspace's plan; you can lower it when creating a key. The ceiling is the plan's maximum. Core keys default to 5,000/min and can be raised up to 10,000/min; Enterprise ceilings are set by contract.

| Plan           | Default (req/min) |                   Ceiling (req/min) |
| -------------- | ----------------: | ----------------------------------: |
| **Core**       |             5,000 |                              10,000 |
| **Enterprise** |            50,000 | 100,000 (custom contracts override) |

Responses include rate-limit metadata:

```http theme={"dark"}
X-RateLimit-Limit: 500
X-RateLimit-Remaining: 498
X-RateLimit-Tier: core
Retry-After: 12   (only on 429)
```

Plan upgrades take effect **immediately**. The gateway re-evaluates the ceiling on every authentication, so there is no need to rotate keys after upgrading.

## Status codes

| Code | Meaning                           | Action                                                                     |
| ---- | --------------------------------- | -------------------------------------------------------------------------- |
| 200  | Success                           | Read the response body                                                     |
| 400  | Malformed request / missing field | Check JSON shape and required fields                                       |
| 401  | Invalid or missing API key        | Re-check the Bearer token                                                  |
| 402  | Insufficient credits              | Top up at [Settings > Cost](https://runinfra.ai/settings/cost#credits)     |
| 403  | Plan blocks the feature           | Upgrade the workspace plan (Core or Enterprise)                            |
| 404  | Model not deployed in workspace   | `GET /v1/models` to list what is available                                 |
| 409  | Idempotency conflict              | Reuse the same idempotency key only for the same request body              |
| 429  | Rate limit exceeded               | Back off per `Retry-After` header                                          |
| 502  | Upstream GPU error                | Retry once; otherwise check [Deployments](https://runinfra.ai/deployments) |
| 503  | GPU worker unavailable            | Retry after 30s. This is usually a cold start.                             |

## SDK compatibility matrix

<Columns cols={3}>
  <Card title="RunInfra TypeScript" icon="code" href="/tools-sdks/runinfra-sdk">
    `npm install @runinfra/sdk`. Native pipeline IDs, typed errors, request IDs, audio, images, and webhook helpers.
  </Card>

  <Card title="RunInfra Python" icon="code" href="/tools-sdks/runinfra-sdk">
    `pip install runinfra`. Native helpers for optimized deployment access.
  </Card>

  <Card title="OpenAI Python" icon="code">
    `pip install openai`. Change `base_url` and `api_key`.
  </Card>

  <Card title="OpenAI JS/TS" icon="code">
    `npm i openai`. Change `baseURL` and `apiKey`.
  </Card>

  <Card title="LangChain" icon="code">
    `ChatOpenAI(base_url=..., api_key=...)`.
  </Card>

  <Card title="LlamaIndex" icon="code">
    `OpenAI(api_base=..., api_key=...)`.
  </Card>

  <Card title="Vercel AI SDK" icon="code">
    `createOpenAICompatible({ baseURL, apiKey })`.
  </Card>

  <Card title="Instructor" icon="code">
    Use it through the OpenAI Python client for supported structured-output chat flows.
  </Card>
</Columns>

## Ready to build?

<Card title="Start building on RunInfra" icon="arrow-right" href="https://runinfra.ai/sign-up" horizontal>
  \$10 in free credits, no credit card required.
</Card>