Skip to main content
An optimization run produces ranked variants of your pipeline across GPUs, quantization schemes, and serving backends, measured against the priority and constraints you set. You pick the winner; RunInfra redeploys in seconds.

​
Optimize and experiment

Choose Optimize when you want RunInfra to rank candidates and apply a winner. Choose Experiment when you want a measure-only bake-off. Experiment results compare candidates side by side, do not declare or apply a winner, and keep deploy and export actions locked until you switch the pipeline to Optimize. The mode and its pricing basis are recorded when the run is admitted. Settlement keeps that recorded basis even if pricing configuration changes while the run is in progress. Experiments settle on measured GPU cost, while Optimize runs keep the value-on-promotion rule quoted for that run.

​
What a run does

Every stage is automatic. You only interact at the start (priority + constraints) and at the end (pick a variant).

​
Deploying the winning variant

Deploying an optimized variant preserves the artifact that won the run. RunInfra carries the selected pipeline files, model revision, serving backend, quantization, chat template, cache settings, speculative decoding settings, and version identity into the deployment request. If the saved artifact snapshot no longer matches the deployable pipeline files, deployment stops before GPU provisioning. This keeps the live endpoint from serving a weaker stock configuration when you expected the optimized build. After a custom image build, the deployment record stores a receipt with the artifact identity and image metadata. Use this receipt when auditing whether a running endpoint came from the selected optimized artifact.

​
Priorities

One dimension ranked first. Other dimensions are reported but do not drive ranking.

​
Constraints

Hard filters. Variants that violate them are dropped before ranking so you never see a “cheap but too slow” false positive at the top.
Combine priority + multiple constraints in one prompt. The agent parses commas as “and” and applies every constraint as an AND filter.

​
Variant types

The optimizer searches these categories when the selected model, modality, runtime, GPU, and workspace support them. You can pin a specific type, but usually should let the optimizer rank compatible options.
The quantized types above are not searched by default. Quantization trades output quality for speed and memory, so it runs only when you ask for it, when the model does not fit the resolved GPU, or when the target is local or on-device hardware. The plan states the decision either way. See Lossless first.
If you have a preference:

​
Serving engine availability

Serving engine selectors keep every option visible. If an engine does not support the selected model type or every stage in a multi-model pipeline, RunInfra disables the option and shows the reason. Unknown model types stay on Auto until RunInfra can verify compatibility. If a model or pipeline edit makes your current engine incompatible, RunInfra switches to a compatible default or Auto and shows one correction notice. This prevents an unavailable engine from remaining selected after the workload changes.

​
Versions and rollback

Each run creates a pipeline version you can compare and roll back to. Rollback is a metadata change; no re-benchmarking needed.
Versions are scoped to one pipeline. Rolling back does not affect running deployments until you explicitly redeploy the reverted version.

​
Compare models, engines, and techniques

Comparisons are measured, not advisory. Ask for a comparison in chat and RunInfra runs real benchmarks before declaring a winner.
Results appear in the Compare Results panel in the session workspace, not as a duplicate table in chat. The panel shows latency, throughput, VRAM, quality, and cost side by side, with the winner highlighted and the execution that produced the measurements identified. Chat keeps the concise completion and final outcome. The underlying comparison evidence rebuilds from persisted results, so the panel and final outcome survive page reloads and older-message pagination. Comparisons stay honest by construction:
  • A model never borrows another model’s baseline. Absent data renders as a gap, not a borrowed number.
  • A candidate that fails to benchmark shows as failed, never as a zero.
  • A candidate without a recorded measurement shows as not measured, never estimated.
  • A winner is only declared when the measured candidates ran on the same GPU. When a comparison would be unfair, RunInfra vetoes the winner and shows the reason.
  • If you cancel mid-comparison, candidates that completed before the cancel keep their measured rows and the run shows as canceled, never as completed.
Comparison runs can also be planned as runbooks: each candidate gets its own lane of phases, and a final verdict phase aggregates the measurements those lanes already recorded without spending additional GPU time.

​
Testing versions

The Test tab can run the current preview graph, the baseline graph, any saved optimization version, or the live deployed endpoint. Baseline tests ignore applied optimization state so you can compare output and latency against the optimized preview or a selected version without accidentally saving the optimized graph first.

​
Compare baseline and optimized

Choose Compare to open baseline and optimized panes around one shared test input. Opening the view does not start compute. Run comparison sends the same input to both runtimes, then each pane reports a concise live state such as waiting, warming, running, measuring, failed, or complete. The final view keeps each output and its measured evidence on the side that produced it. Comparison compute is metered. While a paid comparison is active, RunInfra locks its target, model, input, and parameters so the visible settings cannot drift away from the request being measured. The comparison keeps one durable attempt identity from admission through execution, polling, cancellation, and billing. If a start response or status connection is lost, Retry reconnects with that identity instead of starting a second comparison. Refreshing or reopening the session re-attaches to an active attempt or restores its completed summary. Large modality outputs stay in private storage while the Test tab renders a bounded result summary. Single preview runs remain free. Run comparison bills both the baseline and optimized GPUs at provider cost, including their warm time. After a successful comparison, RunInfra can keep both GPUs warm for 10 minutes of idle time, up to a 30-minute maximum warm session, followed by up to 20 minutes for provider release. Repeating the same comparison within the warm window skips the cold start. The Test tab shows a Warm GPUs countdown. Select End now when you are finished to release both GPUs without waiting for the idle timer.

​
Interruptions, reloads, and cancel

Optimization and runbook executions are durable. The run executes server-side, so refreshing the page, losing your connection, or closing the tab does not stop it.
  • Reload re-attaches live. Reopening the session mid-execution reconnects to the running execution within about a second over a dedicated event stream: phases, live cost, and the Stop control resume updating. A background snapshot poll covers the rare case where the stream cannot connect.
  • Interruptions never strand a run. If an execution is interrupted by a timeout, crash, or redeploy, it converges to a blocked, retryable state with resume and restart actions instead of appearing to run forever.
  • Stop actually stops. Cancel aborts the underlying GPU work and finalizes the run’s billing. A canceled run can never promote an optimization version afterward; work measured before the cancel stays visible as measured results.
  • Honest connection states. If only your view loses its live feed, the cost chip shows Reconnecting or Unavailable instead of a false $0.00. An Unavailable feed means this view lost its connection, not that the run stopped.

​
Settlement: measured or downgraded

Every phase in an accepted plan declares the evidence it must produce before it runs: a benchmark phase declares its measured latency and memory numbers, a quantization phase declares its quality score, a comparison phase declares its verdict rows. When the run settles, RunInfra checks delivered evidence against those declarations.
  • A phase that cannot prove itself downgrades, it does not silently pass. If a declared phase finishes without the outputs it declared, the run still completes. That phase is marked downgraded instead of the whole run failing or the run charging as if the evidence existed.
  • A run that performed no measured work does not read as completed. Downgrades cover a run that measured some things and missed others. A run that produced no measured work at all ends failed and costs nothing. It never settles as a green result with a minimum charge.
  • Downgrades reduce the charge. The portion of the charge tied to a downgraded technique is removed at settlement. You pay for what was measured, not for what was planned.
  • Every downgrade is visible in the run result. The result lists each downgraded phase with its missing outputs and the reason, so a run that reads completed with downgrades is auditable at a glance.
  • Downgraded runs do not promote. A run with downgrades is not marked deployable, and the step that flips an optimized version from draft to ready consults the downgrade list first.
Runs quoted before this contract settle exactly as they were quoted. The measured-or-downgraded check applies to plans accepted after it shipped.

​
Known limitations

Optimization is best-effort. If no variant satisfies your constraints, the run returns zero winners with a recommendation to relax a constraint (usually cost or latency). Failed or cancelled runs cost nothing.
  • Optimization runs use your prepaid credit balance. Every run is quoted before it starts, priced by the models and techniques in the plan, and held against your balance. The estimate you accepted is a ceiling: the settled charge is never higher than the number you said yes to, and a failed or cancelled run releases the whole hold. There are no free optimization runs, at any level of spend. See Pricing and credits.
  • Run duration depends on model size, the number of candidates in the plan, and hardware availability. The session shows live phase progress and live cost instead of a promised finish time.
  • Models above 200 GB on disk are not auto-cached in Instant Start; expect longer cold starts on the winning variant.
  • Quality evidence depends on the selected modality and available eval path. When a measured quality gate is unavailable, RunInfra marks quality as pending or uncharacterized instead of treating an estimated score as a pass.

​
Common questions

A variant is excluded if: a pre-quantized checkpoint does not exist on HF for that (model, quant) pair, the variant violates a hard constraint, or the variant requires a GPU your workspace does not unlock (B200 and H200 are Enterprise).
Every run is quoted before it starts, priced by the models and techniques in the plan, and held against your workspace balance. The charge settles at or below the estimate you accepted, and the rest of the hold returns. A comparison run settles on its measured GPU cost. Token billing applies separately to deployed-endpoint traffic. See Pricing and credits.
Yes. Run optimization against the pipeline (the deployment keeps serving old variant), pick a new winner, click Redeploy. RunInfra rolls replicas one at a time with connection draining; requests in flight complete on the old replica, new ones land on the new variant.
Either the selected GPU/runtime does not support the requested FP8 method, or your constraint set penalized compatible options. The optimizer only offers FP8-family artifacts when the compatibility check passes.
Quality can come from a measured gate for the modality, for example accuracy checks against a baseline. If the gate is unavailable or incomplete, RunInfra shows the quality state as pending or uncharacterized. For task-specific evaluation, use your own test set in the playground and compare variants manually.
Never. Optimization runs happen on isolated benchmark hardware; your deployment serves the previously-chosen variant the whole time. You decide when and whether to switch.
Yes, once the weights are uploaded. Uploading model weights is an Enterprise capability. Upload the model first (Custom models), then run optimization against it like any catalog model.

​
Next steps

Run outcomes

How a run reports a before and after, and when it refuses to publish a ratio.

Lossless first

Why quantization is off by default, and the four conditions that turn it on.

Deploy your pipeline

Turn the winning variant into a live endpoint.

Speculation

Higher decode throughput when a draft model matches your traffic. Gains depend on draft selection and measured acceptance rate, and speculation only ships when it beats the tuned baseline.

GPUs and pricing

Which GPU tiers the optimizer will consider.

Debug bad results

Redirect the optimizer when results don’t fit.