> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Deployments overview
> What happens after an optimization run: a live OpenAI-compatible endpoint you call, or a kit you run on your own hardware.
An optimization run ends in one of two ways. Either RunInfra runs the winning configuration for you as a live endpoint, or you download a kit and run the same configuration yourself.
## Deploy the winner
Your optimization session leaves behind a **pipeline**: the saved setup, holding the model, the settings that won, and the measurements behind them. Deploying turns that into a live endpoint.
```text theme={"dark"}
Deploy this pipeline
```
The agent picks the winning candidate, provisions a GPU, and returns the endpoint URL and key.
Select **Deploy**, then pick the variant and the mode. Provisioning takes 1 to 3 minutes on the first deploy.
RunInfra deploys the exact artifact the selected pipeline version produced: its files, model revision, serving backend, and optimization settings. A missing or stale artifact snapshot is rejected before provisioning rather than replaced with a generic build, so a deployment cannot silently lose the optimization you chose.
## Call the endpoint
The endpoint is OpenAI-compatible. Point any OpenAI client at the RunInfra base URL with your key and the model id the deployment serves.
Set `MODEL_ID` to your deployed model's id from `GET /v1/models` before running these examples.
```python Python theme={"dark"}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.runinfra.ai/v1",
api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)
response = client.chat.completions.create(
model=os.environ["MODEL_ID"],
messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)
```
```typescript TypeScript theme={"dark"}
import OpenAI from "openai";
const modelId = process.env.MODEL_ID;
if (!modelId) throw new Error("Set MODEL_ID to your deployed model's id.");
const client = new OpenAI({
baseURL: "https://api.runinfra.ai/v1",
apiKey: process.env.RUNINFRA_GATEWAY_KEY,
});
const response = await client.chat.completions.create({
model: modelId,
messages: [{ role: "user", content: "Hello" }],
});
console.log(response.choices[0].message.content);
```
```bash cURL theme={"dark"}
curl https://api.runinfra.ai/v1/chat/completions \
-H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
-H "Content-Type: application/json" \
-d "{\"model\":\"$MODEL_ID\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}]}"
```
`GET /v1/models` lists the deployed models your key can reach. See [OpenAI compatibility](/docs/tools-sdks/openai-compatibility) for the full contract, and this is the order every request passes through:
## Two modes
Scale-to-zero. Replicas shed after 5 idle minutes and spin back up on the next request.
| Property | Value |
| - | - |
| Cost | Per-token only, nothing while idle |
| Idle timeout | 5 minutes of no traffic |
| Best for | Development, bursty traffic, cost-sensitive workloads |
Always-on. Replicas stay warm, so no request pays a cold start.
| Property | Value |
| - | - |
| Cost | Per-token rate plus an hourly fee per warm replica |
| Cold start | None |
| Best for | Production APIs with latency commitments |
## Capacity
Each replica serves up to 30 concurrent requests. Managed deploy unlocks when your lifetime paid credit purchases reach \$50.
| Lifetime paid credit purchases | Max replicas | Max concurrent requests |
| - | - | - |
| \$50 or more | 4 | 120 |
Exceeding the budget returns `429` with `Retry-After`. Worker counts persist across stop, start, and GPU changes: an explicit count on the request wins, otherwise the deployment's original sizing, otherwise the workspace default, up to the 4-replica limit.
## The deployment kit
The **Deploy** tab packages the same pipeline version deployment uses, so you can run it on your own hardware. Build it, download it, follow the setup guide, and verify before serving.
The bundle's five load-bearing files are `README.md`, `FETCH_MODEL.sh` which pulls the optimized weights into `./model` from a signed, time-limited URL, `runinfra/manifest.json` carrying provenance and the portability contract, `runinfra/verify.sh` which runs the canary against that manifest, and `runinfra/serve.sh` which starts the selected runtime. It also ships a Dockerfile, a Compose file, benchmark and quality projections, a canary sample request, and SHA-256 checksums. Rebuilding the bundle from the **Deploy** tab issues a fresh signed weight URL.
### How much evidence backs the kit
The manifest carries a `proof_level` so a downstream team can see what the numbers are worth.
| Proof level | Meaning |
| - | - |
| `benchmarked_and_quality_gated` | Benchmarks and a quality gate are present for this artifact. |
| `benchmarked_only` | Benchmarks are present, no quality gate was recorded. |
| `uncharacterized` | The export is usable, but benchmark or quality evidence is missing. |
When proof is incomplete the tab and the manifest show the degraded level rather than filling in confident numbers. A pipeline no run ever measured ships no claim files at all.
### The weights the numbers were measured on
A repository name is a moving target: its default branch advances and tags can be re-pointed, so a kit that names only the repository can load different weights than its own report was measured on.
| What the run resolved | What the kit does |
| - | - |
| An immutable commit | Every launch surface pins it through `HF_REVISION`, and `OPTIMIZATION.lock` records `weights_pin` as `commit` with `weights_reproducible` true. |
| A branch, a tag, or nothing | The kit says so on every buyer-facing file, records `weights_pin` as `moving-ref` with `weights_reproducible` false, and never describes itself as pinned. |
If your kit carries the second case, set `HF_REVISION` to a commit id before relying on its numbers. A kit that already resolved a commit needs no action.
## Manage and test
Test before or after deploying from the Deploy tab playground: real prompts, token counts, latency, variant comparison, no code. Workspaces below the $50 milestone get 100 playground requests a day; at $50 or more it is uncapped. The first request after an idle period on a Flex deployment pays a cold start.
From chat or the [Deployments](https://runinfra.ai/deployments) dashboard you can stop an endpoint (no charges while stopped), start it again (cached weights make restarts fast), or change its GPU tier, and the agent warns when re-optimization is worth doing.
## Common questions
The Deployments dashboard shows `provisioning` and moves to `active` when the replica is serving. The deployment's detail page has logs, so you can confirm the model finished loading.
From the pipeline page, pick any prior optimization variant and redeploy it. Weights are usually still cached from the earlier run, so the rollback is fast.
The first deploy of a pipeline pays the full weight warm-up, which takes minutes. Later cold starts reuse the cached weights and are much faster. If you need no cold start at all, use Active mode.
The request timeout is long but finite. For large `max_tokens`, stream the response instead of waiting for one JSON body.
The Deploy tab names the cause and the next action first. When a deployment starts but never becomes ready, RunInfra reads its startup log and names only what it can prove: a problem fetching the build, storage filling during startup, a GPU runtime incompatibility, a failed start command, or running out of memory. If the log proves none of those, the message stays generic and retryable rather than guessing.
## Related
Point an existing OpenAI client at your endpoint.
Where the pipeline you are deploying came from.
What a deployment costs, and the \$50 milestone.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.