RunInfraby RightNow

Dashboard Sign in Get started

Backed by

Combinator

Optimize any Hugging Face model for production

Paste a model. RunInfra benchmarks the options and picks the winner. Deploy it, or own the stack.

Example workloads

Every optimization ends with a result you can inspect and run.

You get a benchmark receipt and a runnable deployment kit. Nothing hidden.

Serving engine

Compared, not assumed

GPU target

Sized to the model

p95 latency

Benchmarked

Throughput

Measured per GPU

VRAM

Checked for fit

Cost

Tracked per run

GPU kernels

Tuned where supported

Deployment kit

Run it or export it

From prompt to a production stack you own

RunInfra compares, tunes, and benchmarks the stack. Deploy or export it.

Describe a Llama 3.1 70BQwen 2.5 7BDeepSeek V3Mistral 7BPhi-4Gemma 2 9BMixtral 8x7BWhisper Large V3 workload in plain English.

RunInfra compares vLLMSGLangTensorRT-LLMvLLM-Omni and every other engine your model can run on.

It tunes speculative decodingkernel generationserver tuningquantizationKV cache reuseFlashAttention v2continuous batching where it helps, with no config to hand-write.

Deploy on NVIDIA H100NVIDIA H200NVIDIA B200NVIDIA A100NVIDIA L40SNVIDIA L4 and pay per million tokens, or export the stack and self-host.

Model APIs, or build your own stack

Build or export

Optimize any Hugging Face model

Paste any Hugging Face model repo ID.

Compare serving engines
Size the GPU target
Benchmark latency and throughput
Export the deployment kit

Start optimizing

Model APIs

Hosted Model APIs

Pay for input, cached input, and output tokens. See current availability before you choose.

DeepSeek V4 FlashAvailable

DeepSeekDeepSeek V4Flash 0731LLMServerless

deepseek-ai/DeepSeek-V4-Flash-0731

OpenAI-compatible chat completions

per 1M input tokens

per 1M output tokens

1,048,576tokens

Browse all Model APIs

Built and served withNVIDIA

vLLM

SGLang

TensorRT-LLMDockerKubernetes

Deploy on thehardware you choose

Compare real GPU prices and deploy wherever fits. The deployment kit and your own cloud mean no lock-in, no rewrites.

Our cloud, your cloud, or your own hardware

The constant

Every paid optimization ships a deployment kit, so the exit is free.

Managed

RunInfra Cloud

A managed endpoint you do not operate, billed on verified usage.

Your cloud

Your RunPod or Modal

Your account, your endpoint URL, your key.

New Modal connections are paused. The kit's Modal target still deploys with your own token.

More clouds coming soon

Your hardware

Your own machines

The deployment kit runs the same stack anywhere you run containers.

Read the security page

Common questions

Can't find what you're looking for? Get in touch

What is RunInfra?

Describe what you want to run. RunInfra picks compatible open models, benchmarks GPUs, tunes the runtime, and gives you a deploy-ready stack.

Deploy your first optimized model, measured before you ship

Describe the goal. RunInfra builds and optimizes the stack.

Start Building View Pricing

End-to-end encryption

Isolated GPU infrastructure

No training on your data

SOC 2 Type II

RunInfraby RightNow

© 2026 RunInfra. All rights reserved.

Backed by

Combinator

NVIDIA Inception Program

Ask AI about RunInfra

Part of RightNow

From prompt to a production stack you own

RunInfra compares, tunes, and benchmarks the stack. Deploy or export it. Illustrative values, not benchmark results. Published measurements and source records are on the Benchmarks page.

Describe a Llama 3.1 70B workload in plain English.

RunInfra compares SGLang and every other engine your model can run on.

It tunes server tuning where it helps, with no config to hand-write.

Deploy on NVIDIA A100 and pay per million tokens, or export the stack and self-host.

New Session

Optimize Llama-3.1-8B-Instruct on vLLM for cheapest GPU with latency and VRAM checks

Capturing cost-first intent for Llama 3.1 8B on vLLM.

Intake updatedModel, engine, goal

ModelLlama 3.1 8B

EnginevLLM

GoalLow cost, latency checked

Plan drafted3.1s

10 execution phases prepared

Plan readyreview / 10 phases / ~23m

Optimize Llama 3.1 8B on vLLM for cheapest GPU with latency checks

Recommended path: vLLM on L4. Review the plan before execution.

Working...

Runbook generated from the workload

Llama 3.1 8B on vLLM, lowest viable cost

draft

Latency target

p95 under 60ms

VRAM budget

24 GB

Est. runtime

~23 min

Execution plan10 phases, 3 validated

01

AWQ int4 quantizationready

Weight-only int4, calibrated offline

02

FlashAttention v2ready

Fused attention kernels

03

Continuous batchingqueued

In-flight request scheduling

04

Paged KV cachequeued

fp8 cache in paged blocks

05

CUDA graph capturequeued

Replay the decode-step graph

06

Speculative decodingqueued

Draft model proposes tokens

07

Prefix cachingqueued

Reuse shared prompt prefixes

08

Tensor-parallel sizingready

Single GPU, no sharding

09

Warmup and autotunequeued

Lock kernel shapes pre-serve

10

Serving-config tunequeued

Batch size and concurrency

Review the plan, then run.vLLM, L4 to A100

Deploy or export the stack

Pick a target for the optimized L40S build.

ready

TargetSupported GPUs

RunInfra Cloud

selected

Fully managed endpoint, billed per million tokens

H100L40SA100L4

Your RunPod

Deploy to your own RunPod account

H100A100RTX 4090L40S

Your Modal

Deploy to your own Modal workspace

H100A100L40ST4

Generated stackDockerfileserve.shruninfra.yaml

1FROM runinfra/vllm:0.6.3-l40s

2ENV MODEL=Llama-3.1-8B-Instruct

3ENV QUANTIZATION=awq_marlin

4COPY ./weights /models

5CMD vllm serve $MODEL \

6--quantization awq_marlin \

7--kv-cache-dtype fp8 \

8--enable-prefix-caching \

9--max-num-seqs 256 \

10--gpu-memory-utilization 0.92

$0.12 / 1M38ms p95L40S

Own the winning stack before you ship

Not a black box. You get the measured stack to run, deploy, or export.

Supported across the stack

Open models, serving engines, GPUs, and the clouds you deploy to. RunInfra supports every layer of the inference stack.

Models

Engines

GPUs

Clouds