Skip to main content
What this does. Passes a response_format to your deployment so the serving engine constrains the response toward a JSON Schema. RunInfra forwards response_format to the model unchanged. The constraint is applied by the upstream serving engine (for example vLLM guided decoding), not by RunInfra. When to use it. Any downstream consumer that needs a known shape: database inserts, form-filling, tool arguments, data extraction, classification with fixed enums.
Prerequisite: the deployment must support response_format. RunInfra is a passthrough for this field, so JSON-schema enforcement depends on the serving engine and model behind your deployment. Most LLM deployments served by vLLM support guided JSON decoding. If your deployment does not, the model may return a normal completion that the SDK parse() helpers cannot parse, and you should fall back to the raw response_format request below plus your own JSON parsing. Test against your specific model before relying on strict enforcement.

​
Minimal code

The SDK parse() helpers (Python client.beta.chat.completions.parse, TypeScript client.chat.completions.parse) wrap the raw response_format request and validate the result against your Pydantic or Zod model. They are convenient but depend on the deployment honoring the schema. The curl tab shows the raw response_format request, which is the portable form that works with any client and any deployment that supports JSON-schema decoding. 
from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    base_url="https://api.runinfra.ai/v1",
    api_key="YOUR_RUNINFRA_API_KEY",
)

class Receipt(BaseModel):
    merchant: str
    total_usd: float
    date: str
    items: list[str]

response = client.beta.chat.completions.parse(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "Extract the receipt fields."},
        {"role": "user", "content": "I bought coffee and a muffin at Blue Bottle on 2026-04-20 for $12.50"},
    ],
    response_format=Receipt,
)

receipt: Receipt = response.choices[0].message.parsed
print(receipt.merchant, receipt.total_usd)

import OpenAI from "openai";
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

const client = new OpenAI({
  baseURL: "https://api.runinfra.ai/v1",
  apiKey: "YOUR_RUNINFRA_API_KEY",
});

const Receipt = z.object({
  merchant: z.string(),
  total_usd: z.number(),
  date: z.string(),
  items: z.array(z.string()),
});

const response = await client.chat.completions.parse({
  model: "llama-3.3-70b",
  messages: [
    { role: "system", content: "Extract the receipt fields." },
    { role: "user", content: "I bought coffee and a muffin at Blue Bottle on 2026-04-20 for $12.50" },
  ],
  response_format: zodResponseFormat(Receipt, "receipt"),
});

const receipt = response.choices[0].message.parsed;
console.log(receipt.merchant, receipt.total_usd);

curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_RUNINFRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.3-70b",
    "messages": [
      {"role":"system","content":"Extract the receipt fields."},
      {"role":"user","content":"I bought coffee at Blue Bottle on 2026-04-20 for $12.50"}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "receipt",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "merchant":  {"type":"string"},
            "total_usd": {"type":"number"},
            "date":      {"type":"string"},
            "items":     {"type":"array","items":{"type":"string"}}
          },
          "required": ["merchant","total_usd","date","items"],
          "additionalProperties": false
        }
      }
    }
  }'

​
What to tune

ParameterEffect
response_format: { type: "json_object" }Looser mode. Any valid JSON, no schema enforcement. Use for free-form JSON
response_format.json_schema.strict: trueAsks the serving engine to reject any response that does not fit the schema (set by the Pydantic / Zod helpers). Honored only if the deployment supports strict JSON-schema decoding
temperature: 0Best for deterministic extractions. Structured output handles non-zero too

​
Common mistakes

  • Using free-form prompts like “respond as JSON” without response_format. The model can still emit trailing commentary or markdown fences. Always pass response_format.
  • Schemas that allow additionalProperties: true. Strict mode requires additionalProperties: false. The Pydantic / Zod helpers set this for you; raw JSON Schema users must add it.
  • Nested anyOf without discriminators. Use enums or discriminated unions; unbounded anyOf confuses the constrained decoder.
  • Expecting enums to be case-insensitive. Schema enums are exact-match. "Paris" won’t satisfy enum: ["paris", "berlin"].

​
Next steps

Tool calling

Structured args for tool invocations.

Streaming

Stream JSON that parses as it arrives.

RAG

Structured extraction over retrieved context.

API reference

Full response_format contract.