ZYFA: Fast and Accurate FP8 Attention Kernels on AMD MI355X

Zyphra Research released ZYFA, BF16 and FP8 causal prefill attention kernels for AMD's MI355X (CDNA4, gfx950) supporting head dimension 128 and DeepSeek-V2's MLA shape. On activations from Mistral-7B, Qwen3-8B and DeepSeek-V2-Lite at 16K tokens, ZYFA's FP8 kernels reach 1.16–1.76× AITER's throughput, and on Gaussian inputs at 32K tokens 1.26–1.37× at head dimension 128 and 1.99× at the MLA shape. FP8 error is 62–78% lower than AITER's at head dimension 128.