The public method behind every measured claim

What we measure

A published row names its metric before it states a result.

How a number is produced

• Baseline and candidate runs use the same GPU, request protocol, and task definition.
• Serving engine versions stay pinned beside the measurement, so later releases do not rewrite history.
• Benchmark receipts record the run conditions and the date attached to each published claim.
• Quality uses the same protocol and scoring method as its recorded baseline.
• Derived cost stays labeled as derived from the recorded rate and measured throughput.
• Comparison ratios are computed at render time from both absolutes and are never stored.

What we publish and refuse to publish

• We publish measured rows, their conditions, dates, source status, and known absences.
• An unmeasured combination renders an absence. We do not project it or fill it from a vendor table.
• Vendor specifications appear only as labeled citations with a source and retrieval date.
• A cited specification never becomes our measured data.
• We do not present vendor-copied numbers as our measurements.

Versions and dates

• Every measured claim carries an as-of or verified date from its owning evidence.
• Update cadence follows measurement changes, not deploy timing.
• A deploy alone does not advance a measurement date.
• The visible method version changes when this public contract changes.

Reproduction trail

• Every published row keeps a reproduction trail to its recorded conditions.
• Each published package download includes its signed benchmark receipt.
• The receipt records the conditions, engine version, date, and evidence shipped with that package.
• Comparison rows preserve their source conditions verbatim, including workload, hardware, precision, concurrency, and statistic identity.
• Reproduce a row under those conditions, compare both absolutes, then derive any ratio.