> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM optimization runbook

> Serving, quantization, cache, speculation, and expert-parallel optimization for autoregressive language models.

## Stack target

| Technique                        | Trigger                               | Win                             | Plan                 |
| -------------------------------- | ------------------------------------- | ------------------------------- | -------------------- |
| FlashAttention 3                 | Hopper or Blackwell GPUs              | Faster attention kernels        | Core                 |
| FP8 dynamic quantization         | Compatible GPU and model              | Lower latency and memory        | All plans where safe |
| AWQ or GPTQ INT4                 | Compatible checkpoints                | Lower memory footprint          | All plans where safe |
| Chunked prefill                  | Long prompts                          | Smoother time to first token    | All plans            |
| Prefix cache                     | Repeated prompts                      | Faster repeated-prefix requests | All plans            |
| Speculative decoding             | Compatible draft path                 | Lower generation latency        | Core                 |
| MLA backend                      | DeepSeek and Kimi-style architectures | Smaller KV cache                | Core                 |
| Expert parallel                  | MoE models on multi-GPU deployments   | Higher MoE throughput           | Core                 |
| Disaggregated prefill and decode | High-throughput deployments           | Lower tail latency              | Enterprise           |
| Multi-LoRA routing               | Adapter workloads                     | Per-tenant adapter serving      | Core                 |

## Detection, routing, and application

| Decision          | Selection rule                        | Runtime behavior                                       |
| ----------------- | ------------------------------------- | ------------------------------------------------------ |
| Attention backend | Model architecture and GPU family     | Chooses the fastest compatible attention path          |
| Quantization      | Model format, GPU, and quality target | Selects fp8, awq, gptq, or fp16 baseline               |
| Speculation       | Batch size and draft availability     | Uses draft decoding only when a compatible path exists |
| Expert parallel   | MoE model and multi-GPU deployment    | Enables expert-parallel serving where supported        |
| Prefix cache      | Repeated-prefix workload              | Enables cache settings in generated runtime files      |

## Verification

1. The plan should name each selected technique and the reason it applies.
2. The benchmark response should include effective launch settings for attention, quantization, speculation, and cache.
3. The Optimization tab should show measured latency, throughput, cost, and quality for the promoted variant.
4. Generated deployment files should preserve the same runtime settings.

## Rollback

```ts theme={"dark"}
constraints.customerOverrides = {
  skipTechniques: ["speculation"],
};
```

Other common rollback controls:

```ts theme={"dark"}
constraints.customerOverrides = {
  skipTechniques: ["wide_ep"],
};

constraints.customerOverrides = {
  forceQuant: "none",
};
```

Hard bans and quality gates still apply even when a technique is forced.

## Availability notes

* FP8 requires a compatible GPU and backend.
* Speculative decoding depends on a compatible draft path for the selected model.
* Expert-parallel and disaggregated serving depend on deployment topology.
* Multi-LoRA routing depends on adapter availability and runtime support.