> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Lossless first

> RunInfra does not quantize your model by default. Quantization trades output quality for speed and memory, so it only runs when something specific earns it, and the plan states the decision either way.

Quantization makes a model smaller and usually faster, and it does so by giving up some of the model's precision. That is a trade, not a free win. RunInfra treats it as a trade: **quantization is not applied by default.** A run starts from the assumption that you want the model you asked for, at full precision, served as fast as full precision can be served.

## What runs instead

Excluding quantization does not mean excluding optimization. Everything that changes how the model is served, rather than what the model is, still runs:

* Serving engine and runtime selection
* Batching, chunked prefill, prefix caching, block size, and GPU memory utilization tuning
* Lossless FP16 KV cache tuning
* GPU selection and placement
* Speculative decoding, which is output-identical when its equivalence check passes

These are the levers that cost you nothing in output quality, and they are searched first.

## The four conditions that permit quantization

Quantization is enabled when at least one of these is true. Nothing else turns it on.

<Steps>
  <Step title="You asked for it">
    You requested quantization for this run, in the request itself or by selecting a quantized variant to experiment with.
  </Step>

  <Step title="The model does not fit the resolved GPU">
    The model's base-precision weights cannot be served on the GPU the plan actually resolved. This is arithmetic, not preference: without quantization the plan does not run at all.
  </Step>

  <Step title="The target is local hardware">
    The output is aimed at a local device. Local devices are the case where base precision usually cannot be served.
  </Step>

  <Step title="The declared output is a constrained on-device format">
    The deployment target is sized for consumer devices, where base precision usually cannot be served.
  </Step>
</Steps>

<Note>
  A stored preference you set on the workspace always wins, in both directions. If you have explicitly allowed or denied quantization, that decision outranks all four conditions, because it is proven intent rather than an inference.
</Note>

### The fit check is a serving check, not a weights check

Condition two compares against what serving actually needs, not the raw size of the checkpoint. Serving needs the weights plus KV growth and activation headroom, so a checkpoint that is nominally smaller than a card's memory can still fail to fit it. RunInfra uses the same fit arithmetic here that it uses to pick the GPU in the first place, so the decision to quantize can never disagree with the hardware the plan chose.

If any fact needed for that judgement is missing, for example an unknown weight footprint or an unresolved GPU, the check reports that it could not evaluate and the lossless default stays. **A blind spot is never converted into a reason to reduce quality.**

## The plan states the decision either way

You never have to infer what happened. Exactly one of two lines appears in the plan you read before accepting it.

When quantization **is** in the plan, the lane carries a **Quantization basis** line naming the condition that permitted it. For an explicit request it reads:

```text theme={"dark"}
Quantization was applied because it was explicitly requested for this run.
It is not applied by default: it trades model quality for speed and memory.
```

For a hardware fit it names the numbers that failed, the estimated base-precision weight footprint, the VRAM actually needed once serving headroom is included, and the GPU that cannot serve it.

When quantization is **not** in the plan, the caveats section says so and tells you how to change it:

```text theme={"dark"}
Quantization not attempted: it trades model quality for speed and was not
requested. Ask for quantization, or set a hardware target it must fit, to
enable it.
```

A silently applied lossy technique is as dishonest as a silently missing one, which is why both halves of the decision are written down.

## Asking for it

If you want quantization searched, say so:

```text theme={"dark"}
Search for a 4-bit variant of this model
```

```text theme={"dark"}
Quantize this to FP8 and show me the quality delta
```

Or give it a hardware target it has to fit, which lets the fit check earn it on the arithmetic:

```text theme={"dark"}
This has to run on a 24GB card
```

Requesting quantization does not skip the accuracy check. A quantized candidate is measured first and promoted only if its quality gate passes. See [Quality gates](/docs/features/quality-gates).

## Lossy KV cache compression follows the same rule

Compressed KV cache dtypes change attention arithmetic, so they are treated as output-affecting and default to off under the same policy. When they are excluded, the run says:

```text theme={"dark"}
Lossy KV cache compression not attempted: it can reduce model quality and was
not requested. Ask for lossy KV cache compression to enable compressed KV
dtypes. Lossless FP16 KV cache tuning still runs.
```

The last sentence is the important one. The KV lane is not skipped, only its lossy half. Block size and prefix caching are still tuned.

## Related

* [Optimization](/docs/features/optimization) for the full variant search and how winners are ranked.
* [Quality gates](/docs/features/quality-gates) for the accuracy check a quantized candidate has to clear.
* [Run outcomes](/docs/features/run-outcomes) for how a run reports what it changed and what it could not measure.