Model APIs quickstart

This quickstart calls the OpenAI-compatible POST /v1/chat/completions at https://api.runinfra.ai/v1 with a workspace API key.

Using a coding agent? Your agent can set up RunInfra’s open models for you. Copy the prompt from Set up with your coding agent.

Get a workspace API key

Open the Model Library, pick a model that is not marked Paused, and select Get API key. If you already have a key you can copy, the page highlights that key’s copy button instead of making another one.

Put the key in your environment

export RUNINFRA_GATEWAY_KEY="rp_your_key_here"

Make the call

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.runinfra.ai/v1",
    api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)
response = client.chat.completions.create(
    model="nemotron-3-5-lightning-30b",
    messages=[{"role": "user", "content": "Write a TypeScript retry helper."}],
    max_tokens=16384,
)
print(response.choices[0].message.content)

import OpenAI from "openai";

// Server side only: never expose this key in browser code.
const client = new OpenAI({
  baseURL: "https://api.runinfra.ai/v1",
  apiKey: process.env.RUNINFRA_GATEWAY_KEY,
});
const response = await client.chat.completions.create({
  model: "nemotron-3-5-lightning-30b",
  messages: [{ role: "user", content: "Write a TypeScript retry helper." }],
  max_tokens: 16384,
});
console.log(response.choices[0]?.message?.content);

curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nemotron-3-5-lightning-30b",
    "messages": [{"role": "user", "content": "Write a TypeScript retry helper."}],
    "max_tokens": 16384
  }'

What comes back

A normal completion. Token counts and ids are illustrative; reasoning carries the model’s thinking, and content carries the answer.

{
  "id": "chatcmpl-example",
  "object": "chat.completion",
  "model": "nemotron-3-5-lightning-30b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning": "The user wants a retry helper. I should use exponential backoff...",
        "content": "export async function retry<T>(fn: () => Promise<T>, attempts = 3) { ... }"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 14,
    "completion_tokens": 892,
    "total_tokens": 906
  }
}

Check finish_reason before you use content. stop means the model finished on its own. length means it hit the output budget.

List the live models

Run GET /v1/models for the current list; a model that is temporarily paused is reported there and on its Model Library page.

curl https://api.runinfra.ai/v1/models \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY"

How usage is counted

The usage counts in the response are the billed counts; a local tokenizer can count differently. Cached input is part of prompt_tokens, shows in usage.prompt_tokens_details.cached_tokens on models with a cached-input price, and bills at that rate.

Why max_tokens is 16,384

These models reason before they answer, and reasoning tokens are billed output that count toward max_tokens.

You pay for what is generated, not for the ceiling, so a generous ceiling costs nothing extra and a small one can leave you without an answer.

Next steps

Chat completions

Every field the gateway forwards, validates, or drops.

Streaming

Read deltas and ask for the usage frame.

Models

Discover the ids your key can call.

Coding agents

Paste one prompt. Your coding agent sets up RunInfra’s open models for you.

⌘I

Start here

Coding agents

Guides

API reference

Model APIs quickstart

List the live models

How usage is counted

Why max_tokens is 16,384

Next steps

Chat completions

Streaming

Models

Coding agents

​List the live models

​How usage is counted

​Why max_tokens is 16,384

​Next steps

Chat completions

Streaming

Models

Coding agents

List the live models

How usage is counted

Why max_tokens is 16,384

Next steps