Deployments overview

An optimization run ends in one of two ways. Either RunInfra runs the winning configuration for you as a live endpoint, or you download a kit and run the same configuration yourself.

Deploy the winner

Your optimization session leaves behind a pipeline: the saved setup, holding the model, the settings that won, and the measurements behind them. Deploying turns that into a live endpoint.

From chat
From the Deploy tab

Deploy this pipeline

The agent picks the winning candidate, provisions a GPU, and returns the endpoint URL and key.

RunInfra deploys the exact artifact the selected pipeline version produced: its files, model revision, serving backend, and optimization settings. A missing or stale artifact snapshot is rejected before provisioning rather than replaced with a generic build, so a deployment cannot silently lose the optimization you chose.

Call the endpoint

The endpoint is OpenAI-compatible. Point any OpenAI client at the RunInfra base URL with your key and the model id the deployment serves.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.runinfra.ai/v1",
    api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)

response = client.chat.completions.create(
    model="your-deployed-model-id",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.runinfra.ai/v1",
  apiKey: process.env.RUNINFRA_GATEWAY_KEY,
});

const response = await client.chat.completions.create({
  model: "your-deployed-model-id",
  messages: [{ role: "user", content: "Hello" }],
});
console.log(response.choices[0].message.content);

curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"your-deployed-model-id","messages":[{"role":"user","content":"Hello"}]}'

GET /v1/models lists the deployed models your key can reach. See OpenAI compatibility for the full contract, and this is the order every request passes through:

Two modes

Flex
Active

Scale-to-zero. Replicas shed after 5 idle minutes and spin back up on the next request.

Property	Value
Cost	Per-token only, nothing while idle
Idle timeout	5 minutes of no traffic
Best for	Development, bursty traffic, cost-sensitive workloads

Property	Value
Cost	Per-token rate plus an hourly fee per warm replica
Cold start	None
Best for	Production APIs with latency commitments

Capacity

Each replica serves up to 30 concurrent requests. Managed deploy unlocks when your lifetime paid credit purchases reach $50.

Lifetime paid credit purchases	Max replicas	Max concurrent requests
$50 or more	4	120

Exceeding the budget returns 429 with Retry-After. Worker counts persist across stop, start, and GPU changes: an explicit count on the request wins, otherwise the deployment’s original sizing, otherwise the workspace default, up to the 4-replica limit.

The deployment kit

The Deploy tab packages the same pipeline version deployment uses, so you can run it on your own hardware. Build it, download it, follow the setup guide, and verify before serving. The bundle’s five load-bearing files are README.md, FETCH_MODEL.sh which pulls the optimized weights into ./model from a signed, time-limited URL, runinfra/manifest.json carrying provenance and the portability contract, runinfra/verify.sh which runs the canary against that manifest, and runinfra/serve.sh which starts the selected runtime. It also ships a Dockerfile, a Compose file, benchmark and quality projections, a canary sample request, and SHA-256 checksums. Rebuilding the bundle from the Deploy tab issues a fresh signed weight URL.

How much evidence backs the kit

The manifest carries a proof_level so a downstream team can see what the numbers are worth.

Proof level	Meaning
`benchmarked_and_quality_gated`	Benchmarks and a quality gate are present for this artifact.
`benchmarked_only`	Benchmarks are present, no quality gate was recorded.
`uncharacterized`	The export is usable, but benchmark or quality evidence is missing.

When proof is incomplete the tab and the manifest show the degraded level rather than filling in confident numbers. A pipeline no run ever measured ships no claim files at all.

The weights the numbers were measured on

A repository name is a moving target: its default branch advances and tags can be re-pointed, so a kit that names only the repository can load different weights than its own report was measured on.

What the run resolved	What the kit does
An immutable commit	Every launch surface pins it through `HF_REVISION`, and `OPTIMIZATION.lock` records `weights_pin` as `commit` with `weights_reproducible` true.
A branch, a tag, or nothing	The kit says so on every buyer-facing file, records `weights_pin` as `moving-ref` with `weights_reproducible` false, and never describes itself as pinned.

If your kit carries the second case, set HF_REVISION to a commit id before relying on its numbers. A kit that already resolved a commit needs no action.

Manage and test

Test before or after deploying from the Deploy tab playground: real prompts, token counts, latency, variant comparison, no code. Workspaces below the

50 milestone get 100 playground requests a day; at

50 or more it is uncapped. The first request after an idle period on a Flex deployment pays a cold start. From chat or the Deployments dashboard you can stop an endpoint (no charges while stopped), start it again (cached weights make restarts fast), or change its GPU tier, and the agent warns when re-optimization is worth doing.

Common questions

How do I know when the deployment is ready?

The Deployments dashboard shows provisioning and moves to active when the replica is serving. The deployment’s detail page has logs, so you can confirm the model finished loading.

How do I roll back to a previous variant?

From the pipeline page, pick any prior optimization variant and redeploy it. Weights are usually still cached from the earlier run, so the rollback is fast.

My first request took much longer than the rest

The first deploy of a pipeline pays the full weight warm-up, which takes minutes. Later cold starts reuse the cached weights and are much faster. If you need no cold start at all, use Active mode.

A long generation timed out

The request timeout is long but finite. For large max_tokens, stream the response instead of waiting for one JSON body.

The deploy failed

The Deploy tab names the cause and the next action first. When a deployment starts but never becomes ready, RunInfra reads its startup log and names only what it can prove: a problem fetching the build, storage filling during startup, a GPU runtime incompatibility, a failed start command, or running out of memory. If the log proves none of those, the message stays generic and retryable rather than guessing.

OpenAI compatibility

Point an existing OpenAI client at your endpoint.

Optimization runs

Where the pipeline you are deploying came from.

Pricing and credits

What a deployment costs, and the $50 milestone.

Start here

How optimization works

Deployments overview

Deploy the winner

Call the endpoint

Two modes

Capacity

The deployment kit

How much evidence backs the kit

The weights the numbers were measured on

Manage and test

Common questions

OpenAI compatibility

Optimization runs

Pricing and credits

​Deploy the winner

​Call the endpoint

​Two modes

​Capacity

​The deployment kit

​How much evidence backs the kit

​The weights the numbers were measured on

​Manage and test

​Common questions

​Related

OpenAI compatibility

Optimization runs

Pricing and credits

Deploy the winner

Call the endpoint

Two modes

Capacity

The deployment kit

How much evidence backs the kit

The weights the numbers were measured on

Manage and test

Common questions

Related