> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# GPUs and pricing

> RunInfra bills per million tokens, not per GPU hour. Understand how GPU selection, deployment mode, and model size affect your inference cost.

RunInfra charges per million tokens, not per GPU hour. You never interact with infrastructure directly, the agent selects the right GPU for your model automatically based on size, quantization method, and your performance priority. This page explains how token pricing works, which GPUs are available, and how to influence hardware selection when you need to.

## Cost

Inference cost depends on the full pipeline: model, quantization, GPU tier, deployment mode. The Deploy tab projects your actual per-request cost for the exact configuration before you commit. See [Plans](/introduction/plans) for plan-level details and session budgets.

## Available GPU tiers

RunInfra selects from these GPU tiers during the optimization process. The agent matches your model to the right GPU based on model size, quantization method, and your performance priority.

| GPU  | VRAM   | Tier    | Best for                                         |
| ---- | ------ | ------- | ------------------------------------------------ |
| L4   | 24 GB  | Budget  | Quantized 7B-14B models                          |
| L40S | 48 GB  | Mid     | Production 14B-32B, great cost/performance ratio |
| A100 | 80 GB  | High    | 32B FP16, 70B quantized                          |
| H100 | 80 GB  | High    | Production 70B, FP8, TensorRT-LLM                |
| H200 | 141 GB | Premium | 70B FP16, large MoE models                       |
| B200 | 180 GB | Premium | 70B+ FP16, maximum performance                   |

## How GPU selection affects your cost

Larger GPUs serve tokens faster, which changes the economics of your workload in predictable ways:

<Columns cols={3}>
  <Card title="Faster GPU" icon="bolt">
    Lower latency per request. Better for real-time applications where response time matters.
  </Card>

  <Card title="More VRAM" icon="database">
    Supports larger models or higher precision (FP16 instead of quantized). Expands your model options.
  </Card>

  <Card title="Higher tier" icon="arrow-up">
    Better throughput, more requests per second from a single replica.
  </Card>
</Columns>

You pay per token regardless of which GPU runs your model. GPU selection affects latency and throughput, not your per-token rate.

## Deployment modes

The deployment mode you choose affects when you're billed, not how much you pay per token.

<Tabs>
  <Tab title="Flex (Core)">
    **Scale-to-zero.** Your endpoint only runs when it's processing requests. When idle, nothing runs and nothing costs.

    * Pay per token only when requests are being processed
    * Cold starts under 2 seconds on RunInfra Cloud
    * Endpoint scales down after 5 minutes of inactivity
    * Best for development, variable traffic, and cost-sensitive workloads
  </Tab>

  <Tab title="Active (Core)">
    **Always-on.** Your endpoint stays warm 24/7 with zero cold start.

    * Same per-token rate as Flex
    * Flat base fee keeps the endpoint warm at all times
    * Zero cold start latency
    * Best for production APIs with SLA requirements and latency-critical applications
  </Tab>
</Tabs>

## Influencing GPU selection

The agent picks the optimal GPU during optimization. If you want to guide the selection, tell it directly:

```text theme={"dark"}
Use a budget GPU for this, I care about cost
```

```text theme={"dark"}
Use an H100 for maximum performance
```

```text theme={"dark"}
What GPU do you recommend for a 14B model?
```

<Note>
  The agent considers your model size, quantization method, priority, and any hard constraints when recommending hardware. If you specify a GPU tier that is underprovisioned for your model, the agent will warn you before proceeding.
</Note>

## Known limitations

* FP8 availability depends on the exact GPU architecture, runtime, and FP8 method. RunInfra only offers FP8-family artifacts when the compatibility check passes, and marks SM-bound exports accordingly.
* TensorRT-LLM requires a paid Core plan.
* Active mode requires a paid Core plan.

## Common questions

<AccordionGroup>
  <Accordion title="Why do output tokens cost more than input tokens?">
    Output tokens cost real GPU-seconds to generate (autoregressive, one forward pass per token). Input tokens are processed in a single batched pass; they still cost compute, but dramatically less per token. Every major provider prices this way.
  </Accordion>

  <Accordion title="What's the per-second billing granularity?">
    Token billing is per-exact-token, not per-second. Active-mode base fees accrue per-second of warm-replica time. No minimums on either.
  </Accordion>

  <Accordion title="Can I see the GPU tier the optimizer picked?">
    Yes. The variant card in the optimization results shows the GPU for each variant. The Deploy tab shows the active deployment's GPU under Details.
  </Accordion>

  <Accordion title="What if no GPU tier fits my constraints?">
    The optimizer reports "no viable variants" with a recommendation to relax a constraint. Usually raising the cost ceiling or widening the latency ceiling unlocks options. Failed runs do not consume a session.
  </Accordion>

  <Accordion title="Do I pay for the playground?">
    Free (trial) workspaces are capped at 100 playground requests/day. On Core and Enterprise the playground is unlimited and free; we do not charge credits for playground.
  </Accordion>

  <Accordion title="Is there a minimum GPU-time billing on Active mode?">
    Active billing is per-second with a 60-second minimum (so brief starts are rounded up). Once running, every second counts.
  </Accordion>
</AccordionGroup>

## Next steps

<Columns cols={2}>
  <Card title="Plans and sessions" icon="credit-card" href="/introduction/plans">
    Compare the Core and Enterprise plans.
  </Card>

  <Card title="Supported models" icon="cpu" href="/features/models">
    LLMs, speech-to-text, and text-to-speech with pricing by size.
  </Card>

  <Card title="Optimize for cost" icon="zap" href="/features/optimization">
    Use cost priority and hard budget constraints in optimization.
  </Card>

  <Card title="Monitor cost in real time" icon="chart-line" href="/features/monitoring">
    Track spend with daily charts, per-model breakdowns, and alerts.
  </Card>
</Columns>