LLM optimization runbook

Stack target

Technique	Trigger	Win	Plan
FlashAttention 3	Hopper or Blackwell GPUs	Faster attention kernels	Core
FP8 dynamic quantization	Compatible GPU and model	Lower latency and memory	All plans where safe
AWQ or GPTQ INT4	Compatible checkpoints	Lower memory footprint	All plans where safe
Chunked prefill	Long prompts	Smoother time to first token	All plans
Prefix cache	Repeated prompts	Faster repeated-prefix requests	All plans
Speculative decoding	Compatible draft path	Lower generation latency	Core
MLA backend	DeepSeek and Kimi-style architectures	Smaller KV cache	Core
Expert parallel	MoE models on multi-GPU deployments	Higher MoE throughput	Core
Disaggregated prefill and decode	High-throughput deployments	Lower tail latency	Enterprise
Multi-LoRA routing	Adapter workloads	Per-tenant adapter serving	Core

Decision	Selection rule	Runtime behavior
Attention backend	Model architecture and GPU family	Chooses the fastest compatible attention path
Quantization	Model format, GPU, and quality target	Selects fp8, awq, gptq, or fp16 baseline
Speculation	Batch size and draft availability	Uses draft decoding only when a compatible path exists
Expert parallel	MoE model and multi-GPU deployment	Enables expert-parallel serving where supported
Prefix cache	Repeated-prefix workload	Enables cache settings in generated runtime files

The plan should name each selected technique and the reason it applies.
The benchmark response should include effective launch settings for attention, quantization, speculation, and cache.
The Optimization tab should show measured latency, throughput, cost, and quality for the promoted variant.
Generated deployment files should preserve the same runtime settings.

constraints.customerOverrides = {
  skipTechniques: ["speculation"],
};

Other common rollback controls:

constraints.customerOverrides = {
  skipTechniques: ["wide_ep"],
};

constraints.customerOverrides = {
  forceQuant: "none",
};

Hard bans and quality gates still apply even when a technique is forced.