Measured. Verified. Owned.

Real optimization runs on real GPUs, published with the statistic, workload, and provenance next to every number. This page shows what RunInfra optimization buys you and what it means to own your models instead of renting them. Numbers only ship when their stored provenance says measured; the methodology says exactly what we do and do not claim.

Llama 3.1 8B Instruct on NVIDIA A100 (40 GB): 391.76 tok/s at 71.12 ms to first token, $1.489 per 1M output tokens, quality 0.973 vs the FP16 baseline.
Qwen2.5 0.5B Instruct on NVIDIA T4 (16 GB): 1,085 tok/s at 48.82 ms to first token, $0.151 per 1M output tokens, quality 0.982 vs the FP16 baseline.

Llama 3.1 8B Instruct

Continuous-batching serving tuning, plus FP8 dynamic quantization as the promoted winner, measured per configuration

NVIDIA A100 (40 GB), single GPU, measured 2026-07-12

71.12 ms

Time to first token

p50, batched serving (FP16, batch 24)

391.76 tok/s

Output throughput

measured, batched serving (FP16, batch 24)

$1.489

Per 1M output tokens

GPU compute at measured saturation, batched serving config

0.973

Quality vs FP16 baseline

measured, FP8 winner config

Request throughput

requests per second, higher is better

Time to first token, optimized serving

milliseconds, lower is better

Baseline, FP16 serving
Baseline, FP16 serving
Request latencyp50, measured	1665.58 ms
Throughputmeasured	2.24 req/s
VRAMmeasured at load	35.16 GB

RunInfra optimized, measured per configuration
RunInfra optimized, measured per configuration
Time to first token (batched serving)p50, measured	71.12 ms
Time to first token (batched serving)p99, measured	3472.35 ms
Inter-token latency (batched serving)p50, measured	13.2 ms
Output throughput (batched serving)measured	391.76 tok/s
Peak VRAM (batched serving)measured under batch load	37 GB
Capacity (FP8 winner)measured	226 req/min
Cost per request (FP8 winner)measured	$0.000155

RunInfra's automated load test on a single A100-40GB, identical workload on both sides. Serving statistics were measured at batch size 24.

Measurement receipt, quality eval, and limitations

Quality eval:: first-token KL divergence (70%) + perplexity delta (30%) vs FP16 baseline. Score 0.9733 where 1.0 = no detected degradation on this composite under the disclosed workload.
Cost basis:: A100-40GB GPU compute at the rate recorded with the run ($2.10/hr). Excludes idle time, CPU, memory, networking, and platform fees.

Baseline latency is a request-latency percentile; optimized serving latency is time-to-first-token. They are labeled separately and not collapsed into a single reduction claim.
Serving-tuned (FP16, batch 24) and the promoted FP8 winner are different configurations; every number is labeled with its configuration and they are never mixed.
Time to first token p99 under saturated batch load is 3472.35 ms; the p50 to p99 spread is the cost of batching for throughput.
Cost per 1M output tokens is a GPU compute estimate at measured saturation. It excludes idle time, CPU, memory, networking, redundancy, and platform fees.

Qwen2.5 0.5B Instruct

FP8 dynamic quantization (zero calibration) + continuous-batching serving

NVIDIA T4 (16 GB), single GPU, measured 2026-07-11

48.82 ms

Time to first token

p50, FP8 dynamic + batched serving

1,085 tok/s

Output throughput

measured, FP8 dynamic + batched serving

$0.151

Per 1M output tokens

GPU compute at measured saturation, FP8 dynamic + batched serving

0.982

Quality vs FP16 baseline

measured, FP8 dynamic + batched serving

Request throughput

requests per second, higher is better

Time to first token, optimized serving

milliseconds, lower is better

Baseline, FP16 serving
Baseline, FP16 serving
Request latencyp50, measured	793.65 ms
Request latencyp99, measured	797.55 ms
Throughputmeasured	4.2 req/s
VRAMmeasured at load	11.99 GB

RunInfra optimized, FP8 dynamic + batched serving
RunInfra optimized, FP8 dynamic + batched serving
Time to first tokenp50, measured	48.82 ms
Time to first tokenp99, measured	57.66 ms
Inter-token latencyp50, measured	6.95 ms
Output throughputmeasured	1,085 tok/s
Capacitymeasured	509 req/min
Peak VRAMmeasured under batch load	14.38 GB
FP8 quantized request latencyrequest latency, measured; percentile not recorded	150.23 ms

RunInfra's automated load test on a single T4, identical workload on both sides.

Measurement receipt, quality eval, and limitations

Quality eval:: first-token KL divergence (70%) + perplexity delta (30%) vs FP16 baseline. Score 0.9817 where 1.0 = no detected degradation on this composite under the disclosed workload.
Cost basis:: T4 GPU compute at the serverless rate recorded with the run ($0.59/hr). Excludes idle time, CPU, memory, networking, and platform fees.

Baseline latency is a request-latency percentile; optimized serving latency is time-to-first-token. They are labeled separately and not collapsed into a single reduction claim.
Cost per 1M output tokens is a GPU compute estimate at measured saturation. It excludes idle time, CPU, memory, networking, redundancy, and platform fees.
Receipts are added only when runs pass the same provenance gate.

Own your intelligence

The part no per-token price captures. Every cell below is a product fact about the category, not a score.

Deployment freedom comparison between RunInfra, serverless APIs, and managed dedicated endpoints
Capability	RunInfra	Serverless API	Managed dedicated endpoints
Own the optimized model artifacts	Yes, exported to you	No	No, hosted on their infrastructure
Deploy in your own cloud account	Yes, export kits	No	No
Deploy on your own GPUs	Yes	No	No
Run on a laptop (GGUF export)	Yes, where the model fits	No	No
Optimization measured on real GPUs before you pay for serving	Yes, receipts like the one above	Not applicable	Not published
Model keeps working if the provider delists it	Yes, you hold the artifacts	No, catalog changes retire models	Provider dependent
No coding required to optimize and deploy	Yes, agent-driven	API integration required	API integration required

What renting costs

Public list prices from provider pricing pages, cited with date. These are facts for context: pooled per-token rates and a GPU you saturate yourself are different economic units, so we do not chart them against our measured numbers. The receipt above tells you what a model costs to run when you own it.

Serverless per-token list prices, open-weight models
Provider	Offer	List price	Unit	Source
Groq	Llama 3.1 8B Instant	$0.05 in / $0.08 out	per 1M tokens	pricing pageas of 2026-07-12
DeepInfra	Llama 3.1 8B Instruct Turbo	$0.02 in / $0.03 out	per 1M tokens	pricing pageas of 2026-07-12
Groq	Llama 3.3 70B Versatile	$0.59 in / $0.79 out	per 1M tokens	pricing pageas of 2026-07-12
DeepInfra	Llama 3.3 70B Instruct Turbo	$0.10 in / $0.32 out	per 1M tokens	pricing pageas of 2026-07-12
Together AI	Llama 3.3 70BLlama 3.1 8B is no longer listed on Together's pricing page.	$1.04 in / $1.04 out	per 1M tokens	pricing pageas of 2026-07-12

Dedicated GPU rentals and raw cloud instances
Provider	Offer	List price	Unit	Source
Together AI	H100 80GB, dedicated	$5.49	per GPU-hour	pricing pageas of 2026-07-12
Fireworks AI	H100 80GB, on-demand	$7.00	per GPU-hour	pricing pageas of 2026-07-12
DeepInfra	H100 80GB	$2.20	per GPU-hour	pricing pageas of 2026-07-12
Replicate	H100	$5.49	per GPU-hour	pricing pageas of 2026-07-12
AWS EC2	p5.48xlarge (8x H100), us-east-1Whole-instance SKU (about $6.88 per H100-hour as context; single H100s are not sold separately).	$55.04	per instance-hour	pricing pageas of 2026-07-12
Google Cloud	a3-highgpu-8g (8x H100), us-central1Whole-instance SKU (about $11.06 per H100-hour as context).	$88.49	per instance-hour	pricing pageas of 2026-07-12

Raw AWS and Google Cloud GPU instances do run in your own cloud account. The deployment matrix above scopes its third column to managed hosted endpoints, not these raw instances.

We have not measured competitor latency or throughput. Provider numbers on this page are their public list prices, nothing else.
Rented per-token prices and our measured cost per token are different economic units (pooled utilization vs a GPU you saturate), so we never chart them on one scale.
Per-token prices across vendors are not directly comparable: tokenizers differ, and Anthropic's own pricing documentation notes its newer models' tokenizer produces roughly 30% more tokens for the same text (platform.claude.com pricing docs, as of 2026-07-12).
Only modalities with measured, provenance-checked runs appear here. More receipts are added as runs pass the gate, not before.

How we measure

Every performance number on this page comes from a real optimization run on real GPUs, stored with per-metric provenance flags. A number only ships when its flag says measured, its workload is reconstructable, and its statistic is named next to the value.

Baseline and optimized sides run the same automated load test on the same GPU. Latency statistics are labeled per metric: request-latency percentiles for baselines, time-to-first-token and inter-token percentiles for batched serving. We never collapse differently defined statistics into a single reduction claim.

Quality is scored against the FP16 baseline with a composite of first-token KL divergence (weight 0.7) and perplexity delta (weight 0.3), where 1.0 means no detected degradation on this composite under the disclosed workload.

Cost per 1M output tokens is computed by the measurement engine from measured token throughput and the GPU rate recorded with the run. It is a GPU compute estimate at measured saturation and excludes idle time, CPU, memory, networking, redundancy, and platform fees.

Provider prices are their public list prices, linked at the source and dated. We do not measure competitor latency or throughput today; if we ever do, it will be with a disclosed harness, like everything else here.

Changelog

2026-07-12 Added Llama 3.1 8B Instruct on NVIDIA A100-40GB, measured 2026-07-12 with a signed measurement proof.
2026-07-12 First public receipt: Qwen2.5 0.5B Instruct on NVIDIA T4, measured 2026-07-11. Provider list prices captured 2026-07-12.