> ## Documentation Index > Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Run outcomes > Every run ends with a comparable before and after, or with a plain statement of what is missing. RunInfra publishes a speedup only when both sides were measured the same way. A run is only worth paying for if it can tell you what changed. RunInfra ends every run with one of two things: a before and after you can act on, or a statement of exactly which piece of evidence is missing. There is no third option where a number appears without the evidence behind it. ## Two measurements, one instrument A speedup is a ratio, and a ratio is only meaningful when both sides were produced the same way. Two latency numbers taken from different harnesses are not a before and after, they are two unrelated facts. Dividing one by the other publishes the gap between the instruments and calls it your improvement. So every measured result RunInfra records now carries four facts about itself, alongside the number: | Fact | What it records | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | | **Role** | Whether the row is the baseline or an optimized result. Recorded explicitly by whatever measured it, never guessed from a technique name. | | **Latency kind** | The instrument that produced the latency, for example a single-request first-token probe against a concurrent serving benchmark. | | **Throughput unit** | The unit the throughput is in, for example tokens per second against requests per second. | | **Subject** | The model and modality the number belongs to. | Two rows are compared only when all four agree. This is why a run can measure real work and still decline to publish a ratio for it. ## When a comparison is refused A refusal is a correct outcome, not an error. When the evidence does not support a fair comparison, RunInfra crowns no winner and states which axis differed and what the two values were, so you can tell at a glance whether it is a real limitation or something you can fix by re-running. | The comparison is refused when | What the verdict says | | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | The candidates ran on different GPUs | Results are per-GPU measurements, and it asks you to re-run the lanes on one shared GPU target. | | The latency numbers came from different instruments | The rows measure different things, and it names both instruments. | | The per-request costs were derived on different bases | One cost amortized over measured throughput and one derived from single-request latency are different instruments, so no cost winner is crowned. | | Throughput was measured in different units | Throughput measurements with different units are not interchangeable, and it names both units. | | Every ranked candidate resolves to the same stored measurement | There is one number wearing two labels, so there is nothing to compare. | A refused comparison does not fail the run. The measurements it produced stay visible and stay yours. What is withheld is only the ratio that the evidence does not support. ## The baseline is measured, not assumed The baseline a run compares against is measured during that run, on the same instrument as the optimized results, before any technique is applied. It is never synthesized from a catalog figure, an earlier run, or a learned prior. If the baseline measurement did not run, the run has no baseline and says so, rather than substituting a number that would make the ratio look reasonable. A borrowed baseline is the easiest way to manufacture a speedup, and it is the one shortcut this contract exists to close. ## Every run carries a quality verdict Speed without an accuracy answer is half a result. Every completed run now records exactly one quality verdict, and there are three possible values. | Verdict | When it applies | | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Measured** | An output-affecting technique was applied and an accuracy gate recorded a score. The verdict carries the metric, the score, whether it passed, and the techniques it covers. | | **No quality risk** | Nothing output-affecting was applied, so accuracy is unchanged by construction. The applied techniques are named. | | **Not measured** | An output-affecting technique was applied but no accuracy gate recorded a score, so the effect on output quality is unmeasured. The reason is stated. | **A missing verdict is never acceptable.** Not measured is an honest answer and it is written down as one. Silence is not, because a run that skipped the gate would otherwise be indistinguishable from a run that passed it. Two details are worth knowing: * **Quantized KV cache counts as output-affecting**, even though the weights are untouched, because it changes attention arithmetic. * **Speculative decoding does not count**, because it is output-identical when its equivalence check passed, and an unverified one is never promoted. When several stages in one run carry accuracy gates, the **lowest** score decides the verdict. A pipeline is only as accurate as its weakest stage, and reporting the best one would let a passing stage hide a failing one. ## What this means for reading a result * A published speedup means both sides were measured in this run, on one instrument, on one GPU, for one model. Read the quality verdict beside it before treating the speed number as the whole result. * A named refusal means the work happened but the evidence does not support a ratio. Read the named axis to see what to change. * A not-measured quality verdict means treat the speed number as unaccompanied. It is not a failure, and it is not a pass. ## Related * [Quality gates](/docs/features/quality-gates) for how an accuracy check decides promotion, and why a check that could not run is never a pass. * [Optimization](/docs/features/optimization) for how candidates are searched, ranked, and settled. * [Lossless first](/docs/features/lossless-first) for why a run may deliberately apply nothing that could affect quality. * [Benchmarks methodology](/docs/reference/benchmarks-methodology) for how RunInfra reports measurements.