vLLM expands EPD disaggregation for multimodal serving
Start here
vLLM has expanded Encoder/Prefill/Decode (EPD) disaggregation for multimodal serving, adding metadata-only requests that let downstream instances consume remote embeddings without repeating media preprocessing, a dedicated encoder execution path, remote embedding transfer via NIXL and Mooncake, cross-encoder output reuse, and orchestration through vLLM Router, SMG and llm-d. The core capabilities, including MiMo and XPU support, ship in vLLM v0.32.0; metadata-only support for DeepSeek V4.1 Flash is available on main via #62802, and the Router and SMG integrations use their respective main branches. In a DeepSeek V4.1 Flash test on GB200, vLLM reports EPD delivered 6.6–9.5% higher per-GPU throughput and 44–47% lower P90 time-to-first-token than an eight-GPU PD baseline at concurrency 64–128, while per-GPU throughput was lower at concurrency 16.
Generated from reports · updated 11 h ago
Report timeline
Follow the reports to see the event from different sides.
- vLLM BlogvLLM adds encoder disaggregation for multimodal serving with EPD
vLLM has expanded Encoder/Prefill/Decode (EPD) disaggregation for multimodal serving, adding metadata-only requests, a dedicated encoder execution path, remote embedding transfer via NIXL and Mooncake, cross-encoder output reuse, and orchestration through vLLM Router, SMG, and llm-d. Core capabilities including MiMo and XPU support ship in vLLM v0.32.0, while metadata-only support for DeepSeek V4.1 Flash is available on main via #62802. In a DeepSeek V4.1 Flash test on GB200, EPD delivered 6.6–9.5% higher per-GPU throughput and 44–47% lower P90 TTFT than an eight-GPU PD baseline at concurrency 64–128, but had lower per-GPU throughput at concurrency 16.
Heat for this event
Not enough continuous observations yet to draw a trend.