Modal releases DFlash speculators for Qwen 3.5 and 3.6, claiming 5-20% extra speedup
Speculation Is All You Need
เนื้อหานี้ยังไม่มีในภาษาของคุณ จึงแสดงเป็นEnglish
Modal released a set of DFlash draft models for the Qwen series on Hugging Face, including Qwen 3.6 35B-A3B-DFlash and Qwen 3.5 variants from 4B to 122B-A10B, plus a DFlash speculator for Qwen 3.5 397B-A17B built with SGLang. The company says the new drafters add 5-20% speedup over existing DFlash baselines and can run Qwen 3.5 122B-A10B at over 1000 tok/s at concurrency 1 on a B200 node, versus 250 tok/s without speculation. The post argues speculative decoding delivers 2x-3x gains where kernel optimization yields small percentage points, and presents a roofline model for estimating speedup from acceptance length.
Modal explains why it bets on speculative decoding and releases DFlash draft models for the Qwen series, with measured speedups and a roofline model for estimating gains.
แหล่งข่าว:Modal Blog · modal.com