ข้ามไปที่เนื้อหา
vLLM Blog· Inferact, InferMatrix, IBM, and Mooncake·· 2 วันที่แล้วคะแนน AI58

vLLM adds encoder disaggregation for multimodal serving with EPD

Encoder Disaggregation in vLLM: Scaling Multimodal Serving with EPD

เนื้อหานี้ยังไม่มีในภาษาของคุณ จึงแสดงเป็นEnglish

สรุปโดย AI

vLLM has expanded Encoder/Prefill/Decode (EPD) disaggregation for multimodal serving, adding metadata-only requests, a dedicated encoder execution path, remote embedding transfer via NIXL and Mooncake, cross-encoder output reuse, and orchestration through vLLM Router, SMG, and llm-d. Core capabilities including MiMo and XPU support ship in vLLM v0.32.0, while metadata-only support for DeepSeek V4.1 Flash is available on main via #62802. In a DeepSeek V4.1 Flash test on GB200, EPD delivered 6.6–9.5% higher per-GPU throughput and 44–47% lower P90 TTFT than an eight-GPU PD baseline at concurrency 64–128, but had lower per-GPU throughput at concurrency 16.

แหล่งข่าว:vLLM Blog · vllm.ai