Skip to main content
Quantization makes a model smaller and usually faster by giving up some of its precision. That is a trade, not a free win, so RunInfra treats it as one. A run starts from the assumption that you want the model you asked for, at full precision, served as fast as full precision can be served.
Quantization permitoff unless one of these is trueStored preferenceOutranks everything belowAn explicit allow or deny you set on the workspace wins in both directions, because it is proven intent.01You asked for itQuantization requested in the run, or a quantized variant selected to experiment with.02It does not fit the resolved GPUWeights plus KV growth and activation headroom exceed the card the plan resolved.03The target is local hardwareBase precision usually cannot be served on a local device.04The output is an on-device formatThe declared deployment target is sized for consumer devices.LosslessThe default. Engine and runtime selection, batching, prefix caching, lossless FP16 KVtuning, GPU placement, and speculative decoding all still run.Quantization in the planThe lane carries a Quantization basis line naming the condition that permitted it.A fact the check could not evaluate is never converted into a reason to reduce quality.
Quantization permitoff unless one of these is trueStored preferenceOutranks everything belowAn explicit allow or deny you set on the workspace wins in both directions, because it is proven intent.01You asked for itQuantization requested in the run, or a quantized variant selected to experiment with.02It does not fit the resolved GPUWeights plus KV growth and activation headroom exceed the card the plan resolved.03The target is local hardwareBase precision usually cannot be served on a local device.04The output is an on-device formatThe declared deployment target is sized for consumer devices.LosslessThe default. Engine and runtime selection, batching, prefix caching, lossless FP16 KVtuning, GPU placement, and speculative decoding all still run.Quantization in the planThe lane carries a Quantization basis line naming the condition that permitted it.A fact the check could not evaluate is never converted into a reason to reduce quality.

​
Ask for it when you want it

Say so directly:
Or give the run a hardware target it has to fit, and let the arithmetic earn it:
Asking does not skip the accuracy check. A quantized candidate is measured first and promoted only if its quality gate passes.

​
What runs instead

Excluding quantization does not exclude optimization. Everything that changes how the model is served, rather than what the model is, still runs:
  • Serving engine and runtime selection
  • Batching, prefill scheduling, prefix caching, and memory headroom tuning
  • Lossless FP16 KV cache tuning
  • GPU selection and placement
  • Speculative decoding, which is output-identical when its equivalence check passes
These are the levers that cost you nothing in output quality, and they are searched first.

​
The plan says which way it went

You never have to infer what happened. Exactly one of two lines appears in the plan you read before accepting it. When quantization is in the plan, the lane carries a Quantization basis line naming the condition that permitted it:
For a hardware fit it names the numbers that failed instead: the estimated base-precision weight footprint, the VRAM actually needed once serving headroom is included, and the GPU that cannot serve it. When quantization is not in the plan, the caveats say so and tell you how to change it:
A silently applied lossy technique is as dishonest as a silently missing one, which is why both halves of the decision are written down.

​
The same rule covers the rest

Lossy KV cache compression changes attention arithmetic, so compressed KV dtypes are output-affecting and default to off under this same policy. When they are excluded the run says so, and it says that lossless FP16 KV tuning still runs. The KV lane is never skipped, only its lossy half. Non-LLM runs apply the rule per technique family. Every technique declares whether it is lossless by construction or requires measured quality, and a family counts as output-changing when any of its members does. Output-changing families are excluded from the default plan, and the plan’s adjustments section names every family it excluded, so a new technique cannot quietly join the default plan. When a family is permitted, its lane states why it is running, and each candidate still has to pass its own measured quality gate in that modality’s own vocabulary. Where no score was recorded, the run says not measured rather than assuming a pass.

Optimization runs

How candidates are searched and ranked.

Run outcomes

The quality verdict every run carries.

Benchmark reports

Named public benchmarks against a settled winner.