> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Monitoring

> Track requests, latency percentiles, throughput, token usage, and cost across all your RunInfra endpoints from a single real-time dashboard.

RunInfra gives you full visibility into your deployed endpoints through two dedicated pages: the Deployments page for real-time metrics and endpoint management, and the Cost page for cost breakdowns. Together they cover everything you need to understand performance, control spend, and respond to issues without leaving the RunInfra interface.

## Metrics overview

The [Deployments](https://runinfra.ai/deployments) page opens with an overview of real-time metrics for all your endpoints in one place. You can filter by time period (7 days, 30 days, or 90 days) and drill into per-endpoint breakdowns.

<Columns cols={2}>
  <Card title="Requests" icon="chart-bar">
    Total request count, success rate, and error rate. Spot reliability issues at a glance.
  </Card>

  <Card title="Latency" icon="clock">
    Average, P50, P95, and P99 latency. Use percentiles to understand tail latency in production.
  </Card>

  <Card title="Throughput" icon="gauge">
    Requests per second. Monitor traffic spikes and verify your replica count is sufficient.
  </Card>

  <Card title="Tokens" icon="hash">
    Input and output token counts. Useful for capacity planning and cost attribution.
  </Card>

  <Card title="Cost" icon="circle-dollar-sign">
    Per-request and aggregate cost. Track spend against your budget in real time.
  </Card>
</Columns>

## Cost

The [Cost](https://runinfra.ai/settings/cost) page gives you a detailed cost breakdown over the last 30 days.

* **Daily cost chart**. Visualize spend trends and identify unexpected spikes
* **Request volume**. Track how much traffic drove the spend
* **Average request cost**. See blended cost per request across the selected period
* **Cost by model**. See which models contribute most to your bill
* **Credit activity**. Review top-ups, holds, charges, settlements, and refunds

Use this page to understand where your budget is going and which pipelines are worth optimizing further.

## Deployments page

The [Deployments](https://runinfra.ai/deployments) page shows every endpoint at a glance so you can manage the full lifecycle without using the agent.

Each row displays:

* Pipeline name and model
* GPU type and deployment mode (Flex or Active)
* Current status (active, stopped, or provisioning)
* Request count and month-to-date cost
* Quick actions to stop, start, or reconfigure the endpoint

## Metrics retention

How far back you can query metrics depends on your plan:

| Plan       | Retention |
| ---------- | --------- |
| Core       | 1 year    |
| Enterprise | Unlimited |

<Warning>
  Metrics older than your plan's retention window are permanently deleted and cannot be recovered by upgrading later. Export data before downgrading if you need to preserve historical records.
</Warning>

## Exporting metrics

<Tabs>
  <Tab title="CSV from dashboard">
    The Deployments overview and Cost pages have a **Download** button that exports the current filter selection to CSV.
  </Tab>

  <Tab title="Workspace exports">
    For recurring workspace exports, contact RunInfra support. The self-serve metrics export is the dashboard CSV download.
  </Tab>

  <Tab title="Observability tools">
    RunInfra traffic is OpenAI-compatible, so Arize, Langfuse, Helicone, and Datadog pick up RunInfra calls transparently when configured against your existing OpenAI trace pipeline. Point their `base_url` hook at the RunInfra URL and the entire call chain (prompt, response, latency, token counts) streams into your existing dashboards.
  </Tab>
</Tabs>

## Known limitations

* Metrics aggregate at 1-minute granularity. Per-request trace logs are not currently exposed in the dashboard.
* Request/response body logging is opt-in and plan-gated (Core). Default retention for bodies is 24 hours; metadata (status code, tokens, latency) follows the table above.
* Custom alerting (PagerDuty, Slack, email) is Enterprise. For Prometheus or Grafana workflows, contact RunInfra support for the supported integration path on Core and Enterprise plans.

## Common questions

<AccordionGroup>
  <Accordion title="What's the difference between P50, P95, and P99?">
    P50 is the median latency (half of requests were faster). P95 is the 95th percentile (95% were faster). P99 shows tail latency (99% were faster). Optimize against P99 for user-facing chat, P50 for background processing.
  </Accordion>

  <Accordion title="Why are my cost numbers slightly higher than what the Deploy tab predicted?">
    Predicted cost uses your configured model + GPU + mode; actual cost adds the real input/output token counts from traffic, which depend on prompt length and response length. Long conversations and verbose system prompts drive the delta.
  </Accordion>

  <Accordion title="Can I see costs per API key or per user?">
    Per-API-key breakdowns are on Core. Per-end-user attribution requires you to pass the `user` field in your requests; RunInfra groups usage by that ID in the deployments metrics overview.
  </Accordion>

  <Accordion title="How quickly do metrics appear?">
    Requests and latency show within 10 to 30 seconds. Cost numbers refresh hourly to account for rate adjustments and overage calculations.
  </Accordion>

  <Accordion title="Does cold-start time show up in latency?">
    Yes, as part of the first request's latency after a replica spin-up. The metrics overview has a separate **Cold start p99** panel that isolates this for Flex deployments.
  </Accordion>

  <Accordion title="Can I alert on 5xx rate?">
    Enterprise plans get built-in PagerDuty and Slack integrations. All plans can set a Slack webhook manually via the dashboard's Notifications settings.
  </Accordion>
</AccordionGroup>

## Next steps

<Columns cols={2}>
  <Card title="Deploy an endpoint" icon="server" href="/deployments/overview">
    Flex scale-to-zero and Active always-on.
  </Card>

  <Card title="Re-optimize after traffic" icon="zap" href="/features/optimization">
    Tighten constraints once you see real-world latency.
  </Card>

  <Card title="Rate limits" icon="gauge" href="/api-reference/rate-limits">
    Per-key budgets and 429 behavior.
  </Card>

  <Card title="Troubleshoot" icon="life-buoy" href="/tips/troubleshooting">
    Diagnose 4xx, 5xx, and cold-start issues.
  </Card>
</Columns>