vLLM serves DeepSeek-V4.1-Flash on Vera Rubin NVL72 at 58x GB300 NVL72 throughput
DeepSeek-V4.1-Flash on Vera Rubin NVL72: 58x Throughput vs. GB300 NVL72 at 387 tokens/s
vLLM reports serving DeepSeek-V4.1-Flash on a single NVIDIA Vera Rubin NVL72 rack at 1.7M tokens/s per GPU with 387 tokens/s per user, 58x the GB300 NVL72, peaking at 2.52M tokens/s and 1,810 tokens/s for a single user. The gains come from kernels rebuilt for Rubin's split memory domains and FP4 tensor cores, a specialized TP8 decode path, and rack-wide KV and Engram pools shared over NVLink. With prefixes pooled across the rack and Dynamo workload-aware routing, the prefix hit rate holds at 96.5-97% even at high concurrency.
vLLM's third DeepSeek-V4.1-Flash post details how Rubin-specific kernels and rack-wide memory pooling lift throughput 58x over GB300 NVL72, offering a concrete reference for large-scale serving optimization.
Source:vLLM Blog · vllm.ai