> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Structured output
> Constrain JSON responses with response_format. Available on deployments whose serving engine supports JSON-schema decoding.
**What this does.** Passes a `response_format` to your deployment so the serving engine constrains the response toward a JSON Schema. RunInfra forwards `response_format` to the model unchanged. The constraint is applied by the upstream serving engine (for example vLLM guided decoding), not by RunInfra.
**When to use it.** Any downstream consumer that needs a known shape: database inserts, form-filling, tool arguments, data extraction, classification with fixed enums.
**Prerequisite: the deployment must support `response_format`.** RunInfra is a passthrough for this field, so JSON-schema enforcement depends on the serving engine and model behind your deployment. Most LLM deployments served by vLLM support guided JSON decoding. If your deployment does not, the model may return a normal completion that the SDK `parse()` helpers cannot parse, and you should fall back to the raw `response_format` request below plus your own JSON parsing. Test against your specific model before relying on strict enforcement.
## Minimal code
The SDK `parse()` helpers (Python `client.beta.chat.completions.parse`, TypeScript `client.chat.completions.parse`) wrap the raw `response_format` request and validate the result against your Pydantic or Zod model. They are convenient but depend on the deployment honoring the schema. The curl tab shows the raw `response_format` request, which is the portable form that works with any client and any deployment that supports JSON-schema decoding.
```python Python theme={"dark"}
from openai import OpenAI
from pydantic import BaseModel
client = OpenAI(
base_url="https://api.runinfra.ai/v1",
api_key="YOUR_RUNINFRA_API_KEY",
)
class Receipt(BaseModel):
merchant: str
total_usd: float
date: str
items: list[str]
response = client.beta.chat.completions.parse(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "Extract the receipt fields."},
{"role": "user", "content": "I bought coffee and a muffin at Blue Bottle on 2026-04-20 for $12.50"},
],
response_format=Receipt,
)
receipt: Receipt = response.choices[0].message.parsed
print(receipt.merchant, receipt.total_usd)
```
```javascript TypeScript theme={"dark"}
import OpenAI from "openai";
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";
const client = new OpenAI({
baseURL: "https://api.runinfra.ai/v1",
apiKey: "YOUR_RUNINFRA_API_KEY",
});
const Receipt = z.object({
merchant: z.string(),
total_usd: z.number(),
date: z.string(),
items: z.array(z.string()),
});
const response = await client.chat.completions.parse({
model: "llama-3.3-70b",
messages: [
{ role: "system", content: "Extract the receipt fields." },
{ role: "user", content: "I bought coffee and a muffin at Blue Bottle on 2026-04-20 for $12.50" },
],
response_format: zodResponseFormat(Receipt, "receipt"),
});
const receipt = response.choices[0].message.parsed;
console.log(receipt.merchant, receipt.total_usd);
```
```bash curl theme={"dark"}
curl https://api.runinfra.ai/v1/chat/completions \
-H "Authorization: Bearer YOUR_RUNINFRA_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "llama-3.3-70b",
"messages": [
{"role":"system","content":"Extract the receipt fields."},
{"role":"user","content":"I bought coffee at Blue Bottle on 2026-04-20 for $12.50"}
],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "receipt",
"strict": true,
"schema": {
"type": "object",
"properties": {
"merchant": {"type":"string"},
"total_usd": {"type":"number"},
"date": {"type":"string"},
"items": {"type":"array","items":{"type":"string"}}
},
"required": ["merchant","total_usd","date","items"],
"additionalProperties": false
}
}
}
}'
```
## What to tune
| Parameter | Effect |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `response_format: { type: "json_object" }` | Looser mode. Any valid JSON, no schema enforcement. Use for free-form JSON |
| `response_format.json_schema.strict: true` | Asks the serving engine to reject any response that does not fit the schema (set by the Pydantic / Zod helpers). Honored only if the deployment supports strict JSON-schema decoding |
| `temperature: 0` | Best for deterministic extractions. Structured output handles non-zero too |
## Common mistakes
* **Using free-form prompts like "respond as JSON" without `response_format`.** The model can still emit trailing commentary or markdown fences. Always pass `response_format`.
* **Schemas that allow `additionalProperties: true`.** Strict mode requires `additionalProperties: false`. The Pydantic / Zod helpers set this for you; raw JSON Schema users must add it.
* **Nested anyOf without discriminators.** Use enums or discriminated unions; unbounded anyOf confuses the constrained decoder.
* **Expecting enums to be case-insensitive.** Schema enums are exact-match. `"Paris"` won't satisfy `enum: ["paris", "berlin"]`.
## Next steps
Structured args for tool invocations.
Stream JSON that parses as it arrives.
Structured extraction over retrieved context.
Full `response_format` contract.