Skip to content
Zyphra Research·· 24 d agoAI score58

Zyphra Inference Wave 2 brings custom HIP MoE and sparse-attention kernels to MI355X serving

Zyphra Inference Wave 2: Faster MoE and Long-Context Serving on MI355X

AI summary

Zyphra released the second wave of its Zyphra Optimization Engine (ZOE) for AMD Instinct MI355X GPUs, adding four custom HIP kernels for MoE and sparse attention into its production forks of vLLM and SGLang. 40x). 3 checkpoint stays within 3 points of FP8 on four reasoning benchmarks but decodes faster only at high concurrency.

Source:Zyphra Research · zyphra.com