On 2026-07-29, no released or nightly vLLM wheel could serve Kimi K3. We checked. The same day's nightly, vllm 0.26.0, contains zero Kimi K3 files. Support had merged to vLLM main, but nothing you could install carried it. We spent fifteen debugging cycles finding out why, resolved five separate walls on an 8x B300 node, and published the exact configuration that works.
That configuration is now a catalog package. This post explains what we found, what we measured, what we refused to claim, and why we built a package catalog next to an optimization agent.
The catalog
Every card below reads its numbers from the package definition at build time. If a measurement is retracted or re-run, the card changes with it.
Why stock vLLM could not serve it
Five independent things break before Kimi K3 produces a token. Each one is fixable. Nobody had published the set.
- No released vLLM wheel carries K3 support at all.
- Public flashinfer wheels lack the SiTU activation type that K3 mixture-of-experts layers require.
- FlashKDA prefill depends on a private CUDA extension.
- Custom all-reduce fails at the tail of full CUDA graph capture on this node.
- FlashInfer MLA imposes a KV block alignment the defaults do not satisfy.
The package resolves all five with a pinned source build and a proven flag set: tensor parallel 8 with expert parallelism, the Marlin MXFP4 mixture-of-experts backend, the Triton KDA prefill backend, NCCL all-reduce in place of the custom kernel that fails at graph capture, 128-token KV blocks for FlashInfer MLA alignment, and the expandable-segments allocator.
The hardware floor is real
Kimi K3 does not fit on an 8x B200 node. We measured that rather than assuming it. At tensor parallel 8 with expert parallelism the weights alone allocate 175.40 GiB per GPU against 175.75 GiB free of 178.35 GiB total. That leaves nothing for KV cache, activations, CUDA graphs, or the vision tower.
Four configurations were attempted and all four failed. One requested 20 GB of CPU offload with 256 GB of host RAM. The offload flag proved silently ineffective on the mixture-of-experts path and the allocation came back byte-identical. Serving this model needs more than 1.5 TB of HBM inside a single NVLink domain.
What we measured, and what we did not
Every published number came from a single 8x B300 NVLink node at the pinned configuration, on repo-scale coding prompts of 33 to 36K tokens as reported by the server, 2048 output tokens per request, 16 measured requests after 1 warmup, seed 20260727, 40960 context.
Accuracy is stated as a construction argument, not a measured evaluation. Version 1 changes no weight byte and applies no runtime modification, so the served model is the pinned upstream build serving the pinned upstream weights. We publish no cross-server accuracy score because the method is unsound here: a control between two identical stock server instances measured 78.12 percent top-1 agreement, so per-instance reduction order dominates the signal. A number that cannot distinguish a server from itself certifies nothing.
- We did not measure the vision path. K3 ships a MoonViT-V2 tower and its pipeline is image-text-to-text. Every number here is text only.
- We did not evaluate refusal, toxicity, or jailbreak behaviour. The weights are unmodified, so we do not expect a shift, but we did not check.
- We did not measure anywhere near the 1 million token context the model supports.
- The engine wheel these numbers ran on is source-built. The kit ships the pinned build recipe, not the binary.
Why a catalog and not just an agent
Our optimization agent takes your model and your workload and finds a faster way to serve it. That is a process. It runs, it measures, it costs GPU time, and the answer is specific to you.
A catalog package is the opposite shape. The process already ran. What you buy is the finished artifact: the exact serving configuration, the measured proof, and the verifier. No GPU time, no waiting, no agent session. You deploy it on your own hardware and it works, or the receipt says why it should not.
Both exist because they answer different questions. The agent answers what should I do with my model. The catalog answers someone already solved this exact model on this exact hardware, can I have it. For a model like K3, where the hard part was fifteen debugging cycles against five undocumented walls, the second question is worth far more than the first.
What a package contains
- The pinned engine build and the serve flags, written flag for flag into every deploy file.
- A benchmark receipt signed with Ed25519, so the numbers you were sold are the numbers in the artifact.
- A verifier that checks the bytes you downloaded against what we measured.
- For recipe packages, zero weight bytes. The kit pins an upstream Hugging Face revision and your node pulls from the source. We do not store, mirror, or redistribute weights.
The K3 kit is 2.5 MB. The model behind it is about 1.56 TB across 96 shards, pulled from moonshotai/Kimi-K3 at an immutable commit. That asymmetry is the point. You are buying the engineering, not a copy of someone else's weights.
We also do not quantize this model. The MXFP4 weights are Moonshot's own quantization-aware-training release and they arrive untouched. Claiming we quantized it would be the single most damaging false statement available to this package, so the technique field is empty rather than borrowed.
The enterprise tier
Packages above a price threshold now require a signed licence before purchase. The buyer names the licensed entity, the signer, and their authority to bind it, then signs four separate affirmations covering redistribution, resale, and traceability. The signature is recorded append-only with the exact terms text and its hash, and it is bound to the purchase inside the same database transaction that moves the money. An entitlement cannot exist without a signature.
We are direct about what this does and does not do. A serving recipe is configuration and documentation. There is no technical measure that stops a determined buyer from copying it, and we will not pretend otherwise. What the gate provides is a specific, evidenced agreement rather than an implied one.
Where this goes
Frontier open models are getting larger faster than the tooling around them. K3 is 2.8 trillion parameters total with about 104 billion active per token, across 92 mixture-of-experts layers. Day-zero support in a released inference engine is now the exception, not the rule. The window between a model dropping and anyone being able to serve it in production is where the work lives.
We think that window is a product. Not a consulting engagement, not a hosted endpoint you rent forever, but a verified artifact you own and run on your own hardware. Enterprises that cannot send weights to a third party still need someone to have already fought the five walls.
The version 2 plan for this package is already scoped and public: when vLLM pull request 50000 merges and the FlashKDA path becomes publicly buildable, we re-pin, re-measure the same request grid, and publish the new numbers including the prefill path this version runs without. Buyers of version 1 receive it through the maintenance subscription.
Written by
