# RunInfra

> Plain English to production AI inference endpoints. RunInfra selects models, benchmarks GPUs, applies kernel optimizations, and deploys OpenAI-compatible APIs.

## Docs

- [Build with RunInfra](https://runinfra.ai/docs/index.md): Plain English to production AI endpoints.
- [Model APIs quickstart](https://runinfra.ai/docs/api-reference/model-apis-quickstart.md): Call DeepSeek V4 Flash through RunInfra's OpenAI-compatible hosted inference API.
- [What is RunInfra?](https://runinfra.ai/docs/introduction/welcome.md): Two ways in: the chat agent optimizes any open model through a measured session, or call a hosted optimized model on an OpenAI-compatible endpoint.
- [Quickstart](https://runinfra.ai/docs/introduction/quickstart.md): Two first sessions: call a hosted model on an OpenAI-compatible endpoint, or optimize your own model with the agent and deploy the winner.
- [Which model should I use?](https://runinfra.ai/docs/introduction/model-picker.md): Pick the right model for your use case. Decision table by task, size, and performance priority.
- [Pricing and credits](https://runinfra.ai/docs/introduction/plans.md): How RunInfra bills: one workspace balance, credits worth a dollar each, a $10 minimum top-up, what each kind of work costs, the lifetime spend milestones that unlock capabilities, and the spending controls you set.
- [The guided prompt](https://runinfra.ai/docs/prompting/guided-prompt.md): Build a run from the composer menus, or write your own prompt freely. The guided sentence is the default and always describes the run in one place.
- [Prompting best practices](https://runinfra.ai/docs/prompting/best-practices.md): Learn what to include in every RunInfra prompt so the agent builds the right pipeline the first time, without back-and-forth clarification.
- [Example prompts](https://runinfra.ai/docs/prompting/example-prompts.md): Copy-ready prompts for chatbots, summarizers, code generation, multilingual APIs, and more, with notes on what the RunInfra agent builds for each.
- [Debugging](https://runinfra.ai/docs/prompting/debugging.md): Fix common RunInfra issues: wrong model selection, poor optimization results, slow cold starts, and failed deployments, with direct corrective prompts.
- [AI onboarding prompt](https://runinfra.ai/docs/ai-onboarding/prompt-block.md): Copy-paste prompt for any LLM. Teaches your AI assistant how to ship code against RunInfra correctly.
- [Optimization](https://runinfra.ai/docs/features/optimization.md): GPU profiling, quantized-variant search, Forge kernels, and speculation. The RunInfra optimizer picks the right configuration so you don't have to.
- [Lossless first](https://runinfra.ai/docs/features/lossless-first.md): RunInfra does not quantize your model by default. Quantization trades output quality for speed and memory, so it only runs when something specific earns it, and the plan states the decision either way.
- [Quality gates](https://runinfra.ai/docs/features/quality-gates.md): How RunInfra decides whether an optimized candidate is good enough to promote, and why a check that could not run is reported as not evaluated rather than as a pass.
- [Run outcomes](https://runinfra.ai/docs/features/run-outcomes.md): Every run ends with a comparable before and after, or with a plain statement of what is missing. RunInfra publishes a speedup only when both sides were measured the same way.
- [Benchmark reports](https://runinfra.ai/docs/features/benchmark-reports.md): Run named benchmarks against an optimized winner and its baseline from the Benchmarks tab, and see the price before anything starts.
- [Continuous Optimization](https://runinfra.ai/docs/features/continuous-optimization.md): Keep a purchased optimized model package current with newly published versions and measured evidence through an optional monthly subscription.
- [Session workspace](https://runinfra.ai/docs/features/session-workspace.md): Use a pipeline session to run benchmarks, compare candidates, read optimization results, and move between session views.
- [Models](https://runinfra.ai/docs/features/models.md): LLMs, embeddings, rerankers, vision-language, speech-to-text, and text-to-speech models from Hugging Face, with custom weight upload on Enterprise.
- [GPUs and pricing](https://runinfra.ai/docs/features/gpu-pricing.md): RunInfra bills deployed inference per million tokens. Understand how GPU selection, deployment mode, model size, and paid GPU work affect your cost.
- [Monitoring](https://runinfra.ai/docs/features/monitoring.md): Track requests, latency percentiles, throughput, token usage, and cost across all your RunInfra endpoints from a single real-time dashboard.
- [Image generation](https://runinfra.ai/docs/features/image-generation.md): Text-to-image inference on RunInfra: FLUX, SDXL, and Stable Diffusion 3.5 served through a Diffusers FastAPI runtime with torchao FP8 + torch.compile on Ada / Hopper / Blackwell GPUs.
- [Use cases](https://runinfra.ai/docs/use-cases/overview.md): Reference architectures and supported endpoints for voice agents, AI assistants, RAG and semantic search, document AI, and transcription.
- [Deployments overview](https://runinfra.ai/docs/deployments/overview.md): Deploy optimized RunInfra pipelines as production APIs with supported OpenAI-compatible endpoint shapes. Two modes, fast cold starts, and usage-based billing.
- [Deployment failure causes](https://runinfra.ai/docs/deployments/failure-causes.md): When a deployment fails to start, RunInfra reads the worker's own log and names the cause it can prove. When the log proves nothing specific, it says so instead of guessing.
- [Deployment targets](https://runinfra.ai/docs/deployments/targets.md): Where RunInfra can ship your pipeline: managed RunInfra Cloud (default), your own RunPod account (bring your own cloud), self-hosted Modal, or a custom GPU you bring.
- [Deploy to your own RunPod](https://runinfra.ai/docs/deployments/your-runpod.md): Bring your own cloud: deploy an optimized LLM or embedding model to your own RunPod account. You control and pay for the GPU; RunInfra serves the optimized endpoint at zero inference cost once deploy is unlocked.
- [Autoscaling](https://runinfra.ai/docs/deployments/autoscaling.md): How RunInfra replicas scale up and down with traffic. Flex scale-to-zero or Active always-on, with concurrency, queue depth, and cost-latency math.
- [Speculative decoding](https://runinfra.ai/docs/deployments/speculation.md): A small draft model proposes tokens, the target model verifies them in a single pass. Higher throughput with no quality change.
- [Instant Start](https://runinfra.ai/docs/deployments/instant-start.md): RunInfra's weight-caching layer that keeps cold starts fast on scale-to-zero deployments. Covers the cache architecture, eviction rules, multi-GPU sync, and the parts of cold start it does not eliminate.
- [Per-modality optimization runbooks](https://runinfra.ai/docs/runbooks/overview.md): How RunInfra chooses optimization techniques for each model modality and how to verify deployed results.
- [Runbook execution lifecycle](https://runinfra.ai/docs/runbooks/execution-lifecycle.md): What locks when you accept a runbook, how to revise an accepted plan, and what happens to a run that sits parked without activity.
- [LLM optimization runbook](https://runinfra.ai/docs/runbooks/llm.md): Serving, quantization, cache, speculation, and expert-parallel optimization for autoregressive language models.
- [Embedding optimization runbook](https://runinfra.ai/docs/runbooks/embedding.md): How RunInfra optimizes embedding deployments while preserving retrieval quality across runtime selection, batching, gated multi-replica posture, Matryoshka truncation, pooling, normalization, and recall gates.
- [ASR optimization runbook](https://runinfra.ai/docs/runbooks/asr.md): Speech-to-text optimization techniques, quality gates, verification paths, and rollback controls.
- [TTS optimization runbook](https://runinfra.ai/docs/runbooks/tts.md): Text-to-speech optimization techniques, quality gates, verification paths, and rollback controls.
- [Audio-LM and voice-pipeline optimization runbook](https://runinfra.ai/docs/runbooks/audio-lm.md): Optimization targets for audio-language models and co-located voice pipelines.
- [Image generation optimization runbook](https://runinfra.ai/docs/runbooks/image-gen.md): How RunInfra evaluates image generation optimizations for diffusion and DiT pipelines, including precision, compilation, scheduler choice, step distillation, quality gates, and rollback controls.
- [Video generation optimization runbook](https://runinfra.ai/docs/runbooks/video-gen.md): Modal Diffusers hosting, video cache optimization, and production-certification gates for video-generation deployments.
- [Vision-language optimization runbook](https://runinfra.ai/docs/runbooks/vision-language.md): Multimodal prefix cache, concurrency caps, sequence ceilings, and image-bearing benchmarks for vision-language deployments.
- [Load testing optimized deployments](https://runinfra.ai/docs/runbooks/load-testing.md): How to plan high-concurrency tests for each modality and interpret the results.
- [SLO dashboards and regression alerts](https://runinfra.ai/docs/runbooks/slo-alerts.md): Suggested production monitoring for optimized RunInfra deployments.
- [Customer optimization overrides](https://runinfra.ai/docs/runbooks/customer-overrides.md): How to customize optimization choices with force-quantization, skip-technique, budget, latency, and quality controls.
- [Idea to pipeline](https://runinfra.ai/docs/tips/from-idea-to-pipeline.md): Walk through every step of building, optimizing, deploying, and integrating a RunInfra AI pipeline, from blank page to production endpoint.
- [Download a package with the CLI](https://runinfra.ai/docs/tips/download-with-cli.md): Install the RunInfra CLI, approve a terminal from your browser, pull a package your workspace owns, and fetch pinned Hugging Face weights for a recipe package. Covers headless GPU hosts, gated repositories, revoking a terminal, and resuming an interrupted download.
- [Troubleshooting](https://runinfra.ai/docs/tips/troubleshooting.md): Fix common issues with RunInfra pipeline building, optimization, deployment, and API integration, organized by category for fast diagnosis.
- [RunInfra SDK](https://runinfra.ai/docs/tools-sdks/runinfra-sdk.md): Use the native RunInfra TypeScript and Python SDKs for optimized deployment access, scoped API keys, request IDs, replay-safe retries, deployment-gated audio and image helpers, and webhook verification.
- [OpenAI compatibility](https://runinfra.ai/docs/tools-sdks/openai-compatibility.md): RunInfra exposes an OpenAI-shaped HTTP API for verified deployment endpoints. Use supported OpenAI clients with a RunInfra base URL, API key, and deployed model ID.
- [Cookbook](https://runinfra.ai/docs/cookbook/overview.md): Copy-paste recipes for common RunInfra inference patterns. Recipes require a key and a deployment that supports the selected route.
- [Streaming responses](https://runinfra.ai/docs/cookbook/streaming.md): Token-by-token responses with the OpenAI SDK. Server-sent events over HTTPS, same format as OpenAI.
- [Tool calling](https://runinfra.ai/docs/cookbook/tool-calling.md): Function calling with typed arguments. Model picks a tool, you run it, feed the result back. Multi-turn loop.
- [Structured output](https://runinfra.ai/docs/cookbook/structured-output.md): Constrain JSON responses with response_format. Available on deployments whose serving engine supports JSON-schema decoding.
- [Retrieval-augmented generation](https://runinfra.ai/docs/cookbook/rag.md): Embed, retrieve, generate. A complete RAG loop in 30 lines using two RunInfra pipelines.
- [API reference](https://runinfra.ai/docs/api-reference/introduction.md): OpenAI-compatible inference API. Set base_url once, reach verified deployments with a workspace-scoped key or a pipeline-scoped key.
- [Authentication](https://runinfra.ai/docs/api-reference/authentication.md): API key scopes, creation, rotation, and expiration for the RunInfra inference API.
- [Chat completions](https://runinfra.ai/docs/api-reference/chat-completions.md): The exact request contract for POST /v1/chat/completions on hosted Model APIs.
- [Streaming Model APIs responses](https://runinfra.ai/docs/api-reference/streaming.md): Stream OpenAI-compatible chat completion deltas and request client-visible usage.
- [Responses](https://runinfra.ai/docs/api-reference/responses.md): POST /v1/responses, a Responses-shaped compatibility adapter over RunInfra chat-completions deployments.
- [Embeddings](https://runinfra.ai/docs/api-reference/embeddings.md): POST /v1/embeddings, vector embeddings for semantic search, RAG, and clustering.
- [Audio](https://runinfra.ai/docs/api-reference/audio.md): POST /v1/audio/speech and /v1/audio/transcriptions, text-to-speech and speech-to-text.
- [List models](https://runinfra.ai/docs/api-reference/models.md): List the model ids available to a workspace key through GET /v1/models.
- [Rate limits](https://runinfra.ai/docs/api-reference/rate-limits.md): The connection gate, per-key request limits, hosted admission limits, exact 429 bodies, and retry headers.
- [Model APIs limits](https://runinfra.ai/docs/api-reference/limits.md): Context, output, concurrency, and token-rate limits for hosted Model APIs calls.
- [Idempotent retries](https://runinfra.ai/docs/api-reference/idempotent-retries.md): Retry non-streaming Model APIs calls without intentionally starting duplicate inference work.
- [SSE events](https://runinfra.ai/docs/api-reference/sse-events.md): Event shapes for chat streams and optimization sessions: chunks, the usage chunk, terminal error frames, heartbeats, and reconnection.
- [Model APIs errors](https://runinfra.ai/docs/api-reference/errors.md): HTTP statuses, error envelopes, and caller actions for Model APIs.
- [Changelog](https://runinfra.ai/docs/changelog.md): Customer-facing RunInfra releases, feature launches, and platform updates, newest first.
- [Account and access](https://runinfra.ai/docs/faq/account.md): FAQ about sign up, API keys, workspaces, seats, dashboard access, GPUs, regions, data residency, and security.
- [Billing](https://runinfra.ai/docs/faq/billing.md): FAQ about prepaid credits, spend tiers, top-ups, optimization pricing, exact usage settlement, negative balances, and billing guarantees.
- [Models and inference](https://runinfra.ai/docs/faq/models-inference.md): FAQ about supported models, quantization, context windows, streaming, tool calling, and fine-tuning.
- [Glossary](https://runinfra.ai/docs/reference/glossary.md): RunInfra domain terms in one page. GPU, quantization, serving, and agent vocabulary.
- [Benchmarks methodology](https://runinfra.ai/docs/reference/benchmarks-methodology.md): How the numbers on runinfra.ai/benchmarks are measured, labeled, and published. Receipts, provenance flags, the quality composite, the cost basis, and what the page does not claim.
- [Research](https://runinfra.ai/docs/research/overview.md): Open papers from the RunInfra team on attention efficiency, LLM inference, kernel optimization, and the architectures behind production AI infrastructure.

## OpenAPI Specs

- [openapi](https://runinfra.ai/docs/api-reference/openapi.json)