vLLM expands EPD disaggregation for multimodal serving
Nội dung này chưa có bằng ngôn ngữ của bạn; đang hiển thị bằng English.
Bắt đầu từ đây
vLLM has expanded Encoder/Prefill/Decode (EPD) disaggregation for multimodal serving, adding metadata-only requests that let downstream instances consume remote embeddings without repeating media preprocessing, a dedicated encoder execution path, remote embedding transfer via NIXL and Mooncake, cross-encoder output reuse, and orchestration through vLLM Router, SMG and llm-d. The core capabilities, including MiMo and XPU support, ship in vLLM v0.32.0; metadata-only support for DeepSeek V4.1 Flash is available on main via #62802, and the Router and SMG integrations use their respective main branches. In a DeepSeek V4.1 Flash test on GB200, vLLM reports EPD delivered 6.6–9.5% higher per-GPU throughput and 44–47% lower P90 time-to-first-token than an eight-GPU PD baseline at concurrency 64–128, while per-GPU throughput was lower at concurrency 16.
Tạo từ các bài viết · cập nhật 12 giờ trước
Dòng thời gian bài viết
Theo các bài viết để thấy sự việc từ nhiều phía.
- vLLM BlogvLLM adds encoder disaggregation for multimodal serving with EPD
vLLM has expanded Encoder/Prefill/Decode (EPD) disaggregation for multimodal serving, adding metadata-only requests, a dedicated encoder execution path, remote embedding transfer via NIXL and Mooncake, cross-encoder output reuse, and orchestration through vLLM Router, SMG, and llm-d. Core capabilities including MiMo and XPU support ship in vLLM v0.32.0, while metadata-only support for DeepSeek V4.1 Flash is available on main via #62802. In a DeepSeek V4.1 Flash test on GB200, EPD delivered 6.6–9.5% higher per-GPU throughput and 44–47% lower P90 TTFT than an eight-GPU PD baseline at concurrency 64–128, but had lower per-GPU throughput at concurrency 16.
Độ nóng của sự việc này
Chưa đủ quan sát liên tục để vẽ xu hướng.