vLLM expands EPD disaggregation for multimodal serving
เนื้อหานี้ยังไม่มีในภาษาของคุณ จึงแสดงเป็นEnglish
เริ่มตรงนี้
vLLM has expanded Encoder/Prefill/Decode (EPD) disaggregation for multimodal serving, adding metadata-only requests that let downstream instances consume remote embeddings without repeating media preprocessing, a dedicated encoder execution path, remote embedding transfer via NIXL and Mooncake, cross-encoder output reuse, and orchestration through vLLM Router, SMG and llm-d. The core capabilities, including MiMo and XPU support, ship in vLLM v0.32.0; metadata-only support for DeepSeek V4.1 Flash is available on main via #62802, and the Router and SMG integrations use their respective main branches. In a DeepSeek V4.1 Flash test on GB200, vLLM reports EPD delivered 6.6–9.5% higher per-GPU throughput and 44–47% lower P90 time-to-first-token than an eight-GPU PD baseline at concurrency 64–128, while per-GPU throughput was lower at concurrency 16.
สร้างจากรายงาน · อัปเดต 11 ชม. ที่แล้ว
ไทม์ไลน์รายงาน
ไล่ตามรายงานเพื่อดูเหตุการณ์นี้จากหลายด้าน
- vLLM BlogvLLM adds encoder disaggregation for multimodal serving with EPD
vLLM has expanded Encoder/Prefill/Decode (EPD) disaggregation for multimodal serving, adding metadata-only requests, a dedicated encoder execution path, remote embedding transfer via NIXL and Mooncake, cross-encoder output reuse, and orchestration through vLLM Router, SMG, and llm-d. Core capabilities including MiMo and XPU support ship in vLLM v0.32.0, while metadata-only support for DeepSeek V4.1 Flash is available on main via #62802. In a DeepSeek V4.1 Flash test on GB200, EPD delivered 6.6–9.5% higher per-GPU throughput and 44–47% lower P90 TTFT than an eight-GPU PD baseline at concurrency 64–128, but had lower per-GPU throughput at concurrency 16.
ความร้อนแรงของเหตุการณ์นี้
ยังมีการเฝ้าดูต่อเนื่องไม่พอจะวาดแนวโน้ม