Stack target
Vision-language optimization sweeps three serving variants (baseline, balanced, throughput) tuned for image-bearing traffic. Every variant runs the multimodal prefix cache and a capped concurrency limit so image-heavy requests do not OOM.| Technique | Trigger | Win | Plan |
|---|---|---|---|
| Multimodal prefix cache | Repeated image or document inputs | Faster repeated-image prompts | All plans |
| Multimodal concurrency cap | All vision-language deployments | OOM protection for image-heavy requests | All plans |
| Per-family sequence ceiling | Known vision-language family | Safer defaults by model size | All plans |
| Throughput tuning (batch + chunked prefill) | Bulk Q&A or OCR pipelines | Higher batched throughput within OOM-safe limits | All plans |
| Image-bearing benchmark requests | Benchmark sweep | Measures real image-encoder cost | All plans |
Public contract
Vision-language optimization uses vision-specific serving techniques. It does not present generic LLM serving work as runnable vision-language optimization unless the selected runtime can actually apply it to image-bearing requests. RunInfra validates model family, runtime, image support, and benchmark evidence before promoting a vision-language variant.Detection, routing, and application
| Decision | Selection rule | Runtime behavior |
|---|---|---|
| Vision-language modality | Pipeline model and request shape | Uses the vision-language optimization recipe |
| Family detection | Model architecture and config | Chooses family-specific safety caps for Qwen-VL, LLaVA, Mllama, InternVL, Phi-Vision, Idefics, PaliGemma, Pixtral, Molmo, MiniCPM-V, and related families |
| Prefix cache | Every vision-language variant | Enables multimodal prefix caching on all variants |
| Concurrency cap | Model size and image load | Caps concurrent multimodal requests to prevent OOM |
| Throughput tuning | Throughput variant | Raises batch size and chunked prefill within OOM-safe sequence limits |
| Image-bearing benchmark | Benchmark profile | Measures prompt latency with real image inputs |
Verification
- The optimization plan should name the vision-language technique and the selected image workload.
- Benchmarks should include image-bearing prompts rather than text-only substitutions.
- Runtime evidence should preserve multimodal cache, concurrency, and sequence-limit settings.
- Quality checks should preserve vision-specific signals rather than collapsing them into generic text quality.
Rollback
Availability notes
- Per-family sequence ceilings are available for common vision-language families.
- The multimodal prefix cache, concurrency cap, and sequence ceiling apply on every plan. There is no separate paid-tier gate for vision-language serving knobs today.
- OCR-heavy workloads should keep the model at its native precision and rely on the throughput variant for batched gains, since aggressive batching can shift quality on document tasks.