How Modal and Sakana AI served a ~1T-parameter MoE model for the Sakana Namazu API

Modal's forward deployed engineering team worked with Sakana AI to run Sakana Namazu, a ~1T-parameter MoE model tuned for Japanese business context, on Modal for its public API launch. The team applied selective NVFP4 quantization to routed experts only, keeping attention, the router and shared experts in BF16, and calibrated on Japanese-heavy conversations so all 23,040 experts saw representative traffic. A custom DFlash draft model trained on more than 30 billion raw token IDs raised speculative token acceptance by 25% and cut per-token decode latency by about 35%, while a second drafter trained on roughly 1M real requests lifted average acceptance length from 3.51 to 5.01 and cut p99 latency from 68s to 52s.