Managed deployment unlocks once your lifetime credit purchases reach $50. There is no plan to buy: add credits at Settings > Cost and the Deploy tab opens. Before that you can still build, optimize, download kits, call the Model APIs, and test in the playground. See Pricing and credits.
Deploy the winner
Your optimization session leaves behind a pipeline: the saved setup, holding the model, the settings that won, and the measurements behind them. Deploying turns that into a live endpoint.- From chat
- From the Deploy tab
Call the endpoint
The endpoint is OpenAI-compatible. Point any OpenAI client at the RunInfra base URL with your key and the model id the deployment serves.GET /v1/models lists the deployed models your key can reach. See OpenAI compatibility for the full contract, and this is the order every request passes through:
Two modes
- Flex
- Active
Scale-to-zero. Replicas shed after 5 idle minutes and spin back up on the next request.
Capacity
Each replica serves up to 30 concurrent requests, and the replica budget steps with your lifetime credit purchases.
Exceeding the budget returns
429 with Retry-After. Worker counts persist across stop, start, and GPU changes: an explicit count on the request wins, otherwise the deployment’s original sizing, otherwise the workspace default, capped at 32 per deployment.
The deployment kit
The Deploy tab packages the same pipeline version deployment uses, so you can run it on your own hardware. Build it, download it, follow the setup guide, and verify before serving. The bundle’s five load-bearing files areREADME.md, FETCH_MODEL.sh which pulls the optimized weights into ./model from a signed, time-limited URL, runinfra/manifest.json carrying provenance and the portability contract, runinfra/verify.sh which runs the canary against that manifest, and runinfra/serve.sh which starts the selected runtime. It also ships a Dockerfile, a Compose file, benchmark and quality projections, a canary sample request, and SHA-256 checksums.
How much evidence backs the kit
The manifest carries aproof_level so a downstream team can see what the numbers are worth.
When proof is incomplete the tab and the manifest show the degraded level rather than filling in confident numbers. A pipeline no run ever measured ships no claim files at all.
The weights the numbers were measured on
A repository name is a moving target: its default branch advances and tags can be re-pointed, so a kit that names only the repository can load different weights than its own report was measured on.
If your kit carries the second case, set
HF_REVISION to a commit id before relying on its numbers. A kit that already resolved a commit needs no action.
Manage and test
Test before or after deploying from the Deploy tab playground: real prompts, token counts, latency, variant comparison, no code. Workspaces below the 50 or more it is uncapped. The first request after an idle period on a Flex deployment pays a cold start. From chat or the Deployments dashboard you can stop an endpoint (no charges while stopped), start it again (cached weights make restarts fast), or change its GPU tier, and the agent warns when re-optimization is worth doing.Common questions
How do I know when the deployment is ready?
How do I know when the deployment is ready?
The Deployments dashboard shows
provisioning and moves to active when the replica is serving. The deployment’s detail page has logs, so you can confirm the model finished loading.How do I roll back to a previous variant?
How do I roll back to a previous variant?
From the pipeline page, pick any prior optimization variant and redeploy it. Weights are usually still cached from the earlier run, so the rollback is fast.
My first request took much longer than the rest
My first request took much longer than the rest
The first deploy of a pipeline pays the full weight warm-up, which takes minutes. Later cold starts reuse the cached weights and are much faster. If you need no cold start at all, use Active mode.
A long generation timed out
A long generation timed out
The request timeout is long but finite. For large
max_tokens, stream the response instead of waiting for one JSON body.The deploy failed
The deploy failed
The Deploy tab names the cause and the next action first. When a worker starts but never comes up, RunInfra reads that worker’s own log and names only what it can prove: an image or registry problem, storage filling during startup, a GPU runtime incompatibility, a failed entrypoint, or an out-of-memory kill. If the log proves none of those, the message stays generic and retryable rather than guessing.
Related
OpenAI compatibility
Point an existing OpenAI client at your endpoint.
Optimization runs
Where the pipeline you are deploying came from.
Pricing and credits
What a deployment costs, and the $50 milestone.