vLLM Blog· Inferact, InferMatrix, IBM, and Mooncake·2026-11-11 08:00· 2 d agoAI score58

vLLM adds encoder disaggregation for multimodal serving with EPD

Encoder Disaggregation in vLLM: Scaling Multimodal Serving with EPD

AI summary

vLLM has expanded Encoder/Prefill/Decode (EPD) disaggregation for multimodal serving, adding metadata-only requests, a dedicated encoder execution path, remote embedding transfer via NIXL and Mooncake, cross-encoder output reuse, and orchestration through vLLM Router, SMG, and llm-d. Core capabilities including MiMo and XPU support ship in vLLM v0.32.0, while metadata-only support for DeepSeek V4.1 Flash is available on main via #62802. In a DeepSeek V4.1 Flash test on GB200, EPD delivered 6.6–9.5% higher per-GPU throughput and 44–47% lower P90 TTFT than an eight-GPU PD baseline at concurrency 64–128, but had lower per-GPU throughput at concurrency 16.

Source:vLLM Blog · vllm.ai

Multimodal Open source Deployment engineering Tutorials #Tutorials / practice #Deployment / engineering #Multimodal #Open-source ecosystem