Skip to main content
A deployment turns an optimized pipeline into a live HTTP endpoint. You get a URL, a scoped API key, autoscaling replicas, Instant Start cold-start caching, and supported OpenAI-compatible routes where the deployment exposes them.
Deployment unlocks once your lifetime credit purchases reach $50. There is no plan to buy: add credits at Settings > Cost and the Deploy tab opens. Before that you can still build, optimize, call the Model APIs, and test in the playground. See Pricing and credits.

Request lifecycle

Every request hits the edge, authenticates against your API key, routes to a replica with headroom, and streams tokens back. Cold starts pull from Instant Start; warm requests skip it.

Where it runs

Four deployment targets, picked when you click Deploy: See Deployment targets for the full picking guide.

Deploy

The agent picks the winning variant, provisions a GPU, and returns the endpoint URL and key.
RunInfra uses your intake choices to order the deployment paths. If you chose a local machine, edge device, own server, or on-premises target, Your hardware appears first and shows the captured device memory budget. Local and edge exports default to Local GPU. Own-server and on-premises exports default to Container. Cloud-focused intake keeps managed cloud first. Your manual target choice remains selected after the session finishes loading.

Optimized artifact deploys

When you deploy an optimized variant, RunInfra Cloud uses the deployable artifact from the selected pipeline version. The deploy request carries the pipeline files, model revision, serving backend, and optimization settings that produced the winning variant. Managed deployments reject missing or stale artifact snapshots before provisioning. This prevents a deployment from silently falling back to a generic base image and losing the optimization that you selected. Each custom artifact build returns a deployment receipt with: The Deploy tab refreshes the live capability contract before provisioning. If a backend or modality is not currently enabled, deployment stops before creating GPU resources and shows the required action.

Modality verification and API keys

Managed deployments verify through a modality-native gateway canary before they go active. Voice pipeline, speech-to-text, text-to-speech, audio-language, and vision-language deployments verify this way and can mint working API keys immediately after the deployment verifies; previously that immediate key flow covered LLM and embedding deployments only. Minted keys pair with the flat base URL https://api.runinfra.ai/v1. Image generation, rerank, and classification managed deployments remain unavailable. Rerank and classification are delivered through deployment kits instead; see Deployment kits.

Deployment kits

The Deploy tab packages the same selected pipeline version that deployment uses. Its Deployment kit area centers on one optimized pipeline package: build it, download it, run the setup guide, and verify before serving. Downloads include these files: Runtime files are generated for the selected backend. vLLM and SGLang launch their Python serving modules, vLLM Omni pins the audio runtime, TensorRT-LLM uses Triton Server, and Diffusers or TEI preserve the base image entrypoint and required environment. Rerank and classification kits use the explicit tei export format: the generated serving setup runs text-embeddings-inference, and the kit’s serve script answers the kit’s own documented endpoints, /rerank for rerank kits and /predict for classification kits. The manifest exposes a proof_level so downstream teams can see how much measured evidence backs the export: When proof is incomplete, the Deploy tab and generated manifest show the degraded proof level instead of filling in confident numbers. Missing benchmark, quality, library-version, or out-of-envelope data is rendered as uncharacterized. Export job status preserves the selected optimizationVersionId and runbook executionId. Use those ids to confirm the downloaded bundle or Hugging Face push belongs to the accepted optimization run before treating it as production evidence. The signed weight URL in FETCH_MODEL.sh expires. If the link is stale, rebuild the bundle from the Deploy tab to get a fresh URL. Before building a deployment kit, RunInfra scans customer-authored and edited files for sensitive values. Files that exactly match the current generated Code artifact pass without being misclassified because they contain safe placeholders, environment variable names, or setup instructions. If the generated snapshot is stale, the Deploy tab asks you to sync or regenerate the Code artifact. If a customer-added or edited file contains a real secret, export still stops and identifies the affected file paths without exposing the secret value.

The kit pins the weights the run measured

A model repository name is a moving target. Its default branch advances and tags can be re-pointed, so a kit that names only the repository loads whatever that repository holds on the day you run it, while the kit’s own report quotes numbers measured on some earlier state. When the run resolved the exact weights revision it measured, every generated launch surface pins it. The serve script, the container build, the Compose file, the Kubernetes manifests, and the managed-cloud targets all carry the revision through an HF_REVISION setting, passed to each serving backend with that backend’s own revision flag. The revision that gets pinned is the commit the run actually loaded, recorded by the worker, not merely the reference the run asked for. The applied-optimization report states it plainly, pinned to commit <sha>, the weights this run measured, and OPTIMIZATION.lock records weights_pin as commit with weights_reproducible true. When the run did not resolve an immutable commit, the kit says so rather than leaving a missing field to be read as reproducibility. Every buyer-facing surface, README.md, OPTIMIZATIONS.md, APPLIED-OPTIMIZATION.md, and optimization-report.md, carries this banner:
A recorded branch or tag gets its own banner, because a branch is a serving instruction and not a pin:
The launch files still pass that branch, because it is what the run asked for, but OPTIMIZATION.lock records weights_pin as moving-ref with weights_reproducible false, and nothing in the kit describes it as pinned.
The weights pin and the optimization claim are separate facts. A pipeline no run ever measured ships no claim surfaces at all: no OPTIMIZATIONS.md, APPLIED-OPTIMIZATION.md, OPTIMIZATION.lock, or optimization-report.md. Its README.md still states the missing pin.
Before relying on a kit’s numbers, do what the banner asks: set HF_REVISION to an immutable commit id of the model repository. A kit that already carries a resolved commit needs no action.

Deployment modes

Scale-to-zero. Replicas shed after 5 idle minutes, spin back up in under 2 seconds on next request.
DEPLOYMENT MODESDeploy1 to 3 minutes on the first deployservingreplica is servingscale-to-zeroafter 5 idle minutesInstant Startunder 2 seconds on next requestFirst deploy of a pipeline pays the full weight warm-up (minutes).Subsequent cold starts reuse Instant Start’s weight cache and are much faster.
DEPLOYMENT MODESDeploy1 to 3 minutes on the first deployservingreplica is servingscale-to-zeroafter 5 idle minutesInstant Startunder 2 seconds on next requestFirst deploy of a pipeline pays the full weight warm-up (minutes).Subsequent cold starts reuse Instant Start’s weight cache and are much faster.

Calling the endpoint

OpenAI-compatible. Use the OpenAI Python or JavaScript SDK by setting the RunInfra base URL and API key:
Verified clients include the OpenAI Python and JavaScript SDKs, LangChain, LlamaIndex, Vercel AI SDK, Instructor, and direct HTTP for supported endpoints. See OpenAI compatibility for the full contract.

Capacity

Each replica serves up to 30 concurrent requests. Replica budgets step with your lifetime credit purchases: Exceeding the budget returns 429 with Retry-After. See Autoscaling to raise caps and tune concurrency.

Playground

Test before or after deploying from the Deploy tab. Send prompts, inspect token counts, compare variants, measure latency, without writing a line of code. After deployment, the playground targets the selected endpoint row, so the latency and output come from the endpoint you are inspecting.
The first playground request after an idle period pays a cold start (up to 2 s on Flex deployments). Subsequent requests are fast.

Manage endpoints

From chat or Deployments:

Stop

Pause the endpoint. No charges while stopped.

Start

Resume from stopped. Cached weights make restarts fast.

Change GPU

Switch tier. The agent warns if re-optimization is recommended.
Worker counts persist across these actions. The worker count requested at deploy time is reapplied on restart, start, and GPU changes: an explicit count on the request always wins, otherwise the deployment’s original sizing applies, otherwise the workspace default. Worker counts are capped at 32 per deployment, and your spend tier or an operational cap can be lower. See Pricing and credits for the replica and GPU ceiling at each tier.

Known limitations

  • Request timeout is long but finite. For large max_tokens, stream the response so you don’t hit the timeout.
  • First deploy of a pipeline pays the full weight warm-up (minutes). Subsequent cold starts reuse Instant Start’s weight cache and are much faster.
  • Active mode unlocks with deploy, at $50 in lifetime credit purchases.

Common questions

The Deployments dashboard shows provisioning and transitions to active when the replica is serving. You can watch logs from the deployment’s detail page to confirm the model finished loading.
The playground under the Deploy tab is the preview. Send real prompts, inspect quality and latency, compare variants, before you commit to deploying an endpoint that serves external traffic.
From the pipeline page, pick any prior optimization variant and redeploy it. Weights are usually still cached from the earlier run, so the rollback is fast.

Next steps

Deployment targets

Managed RunInfra Cloud, your RunPod, self-hosted Modal, or custom GPU.

Autoscaling

Replica budget, concurrency, Flex vs Active knobs.

Instant Start

Cold-start weight caching explained.

Speculation

Draft-model speculative decoding for throughput.