RunInfra
Own your AI. We benchmark GPUs, tune supported runtime paths, and deploy compatible open-source models with verified evidence.
Start building
Ask AI about RunInfra
Personal AI assistants on the models you own.
Llama, Hermes, Qwen, or supported open Hugging Face chat models. Tool use, policy, and streaming on a single GPU.
Chat
Tool use
Code agent
Knowledge work
What you actually own
The optimization knobs, the codebase, the model choice. None of it locked away.
01
Pick the model your product needs.
Llama, Mistral, Qwen, Phi, Gemma, or your own fine-tune. The agent retunes the kernels around your choice.
02
Tool use and streaming on your stack.
OpenAI-compatible chat completions with tool calling and streaming. No proprietary client.
03
Own the runtime and the data.
Export the Dockerfile and serve script. Run on managed RunInfra or your own GPUs.
Three ways to ship a chat assistant
Most teams pick between speed and control. RunInfra keeps both in one workflow.
| What matters | RunInfraRecommendedFast path with model control and export. | Closed chat APIsPer-token, hosted. | DIY self-hostingFull control, heavy operations. |
|---|---|---|---|
| 01Launch | Pick model, optimize, deploy Start quickly and keep the production path open. | Call provider endpoint Fast first demo, but the runtime stays rented. | Build serving stack first Infrastructure work comes before product learning. |
| 02Model control | Bring the model ID Keep model choice and serving decisions visible. | Provider catalog You use what the provider exposes. | Your model Full control if your team maintains the runtime. |
| 03Tuning | Measured latency and GPU cost Compare serving choices before deployment. | Opaque Latency and batching stay behind the API. | Manual profiling Your team owns tuning and regressions. |
| 04Export | Managed now, export when needed Use the endpoint first and take the deploy package later. | Locked endpoint You keep calling the provider. | Already owned Export exists because you built everything yourself. |
| 05Operations | Low until you choose to own it Operate managed, then export with the same measured plan. | Low, with lock-in Less infra work, less production control. | High You own infra, failures, upgrades, and serving changes. |
| 06Security | SOC 2 Type 2 Audited controls across access, logging, and incident response. | Varies by vendor Compliance depends on the third party sitting in the request path. | You build it Your team owns the audit trail, logging, and access controls. |
RunInfra
RecommendedFast path with model control and export.
Launch
Pick model, optimize, deploy
Start quickly and keep the production path open.
Model control
Bring the model ID
Keep model choice and serving decisions visible.
Tuning
Measured latency and GPU cost
Compare serving choices before deployment.
Export
Managed now, export when needed
Use the endpoint first and take the deploy package later.
Operations
Low until you choose to own it
Operate managed, then export with the same measured plan.
Security
SOC 2 Type 2
Audited controls across access, logging, and incident response.
Code you own. Deploy anywhere.
The full recipe ships with you. Codebase, kernels, engine config, weights. Run it anywhere.
Live
Try Mistral 7B and rerun the harness...
Tool definitions3
runinfra-ai-assistant/
models/
evals/
Managed RunInfra
Our GPUs, per-million-tokens billing from L4 to B200.
Your infrastructure
AWS, GCP, RunPod, bare metal. Same Dockerfile, your cluster.
Local workstation
docker compose up. Full pipeline on a single GPU.
Supported HF chat models work
Supported chat-capable models on Hugging Face run through the compatible recipe. Search the live catalog above. The examples below are just a starting view.
HF
What RunInfra tunes
Every stage of the pipeline, retuned per model and GPU.
Kernel sweep
FlashAttention, FlashInfer, Marlin, custom fusion. Correctness + speedup gates on every rewrite.
Quantization
FP8, AWQ, GPTQ, HQQ, INT8 SmoothQuant, NVFP4. Quality scored against your accuracy floor.
KV cache
FP8, INT4, and TurboQuant 3-4 bit KV compression. 60 to 75% VRAM savings.
Speculative decoding
EAGLE3, MTP, n-gram lookup, draft model. 1.3 to 2x decode speedup, weights untouched.
Serving config
Continuous batching, chunked prefill, PagedAttention, prefix cache. Tuned across vLLM, SGLang, TensorRT-LLM.
Multi-cloud capacity
Pareto GPU selection across L4 to B200. Managed or exportable.
Try this pipeline
Edit the model, engine, or GPU inline. Send to retune the stack in the dashboard.
Common questions
Can't find what you're looking for? Get in touch
Can I bring my own fine-tuned model?
Yes, for supported chat-capable models. Paste the Hugging Face ID; unsupported or untuned shapes show baseline-only or gated status instead of a tuned run.
