> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# SLO dashboards and regression alerts

> Suggested production monitoring for optimized RunInfra deployments.

## What to monitor

Production teams should monitor both endpoint performance and optimization quality. The key question is simple: did the deployed pipeline keep the latency, throughput, cost, and quality profile that won the optimization run?

| Signal                  | What it means                                                   | Alert when                                                   |
| ----------------------- | --------------------------------------------------------------- | ------------------------------------------------------------ |
| Runtime drift           | Runtime launch settings differ from the optimized configuration | Any confirmed drift affects production traffic               |
| Technique fallback      | A selected optimization technique could not be applied          | The rate rises above normal for a model family or runtime    |
| Quality rollback        | A candidate regresses below the configured quality gate         | Rollbacks cluster on the same model, runtime, or technique   |
| Per-phase cost outlier  | An optimization phase costs much more than expected             | p95 phase cost rises materially above the trailing baseline  |
| Optimization abort rate | Sessions fail or refund after work starts                       | The rate spikes across unrelated workspaces                  |
| Plan-limit hints        | Users hit plan gates while optimizing                           | The rate changes enough to affect conversion or support load |

## Per-modality SLO examples

Tune these thresholds for your own traffic and model mix.

| Modality         | SLO                                          | Alert threshold            |
| ---------------- | -------------------------------------------- | -------------------------- |
| LLM, 7B-class    | p95 first token under 200 ms                 | Above 300 ms for 5 minutes |
| LLM, 70B-class   | p95 first token under 400 ms                 | Above 600 ms for 5 minutes |
| LLM throughput   | More than 50 RPS per H100-class deployment   | Below 40 RPS for 5 minutes |
| Embeddings       | p95 vector latency under 50 ms at batch 64   | Above 100 ms for 5 minutes |
| ASR              | Real-time factor under 0.05 at batch 16      | Above 0.10 for 5 minutes   |
| TTS              | Streaming time to first byte under 200 ms    | Above 500 ms for 5 minutes |
| Image generation | FLUX Schnell under 1.5 s per image           | Above 2.5 s for 5 minutes  |
| Vision-language  | Repeated-image time to first token under 1 s | Above 2 s for 5 minutes    |

## Dashboard layout

A compact matrix works well for operations teams:

```text theme={"dark"}
                    LLM   Embedding   ASR   TTS   Image-gen   VL
p95 latency          .      .         .     .     .           .
Throughput           .      .         .     .     .           .
Quality              .      .         .     .     .           .
Drift events         .      .         .     .     .           .
Fallback rate        .      .         .     .     .           .
Cost                 .      .         .     .     .           .
```

Use green for healthy, yellow for approaching the threshold, and red for active breach. Keep one row per metric, one column per modality, and one cell per latest evaluation window.

## Alert routing

Start with three alert classes:

* **Customer-facing incident**: endpoint unavailable, severe latency breach, or repeated upstream failures.
* **Optimization quality risk**: drift, fallback spike, or quality rollback cluster.
* **Cost control risk**: phase cost outliers or unexpected reservation failures.

Include the deployment ID, model ID, optimization version, request ID where available, and the last known runtime settings in each alert.