Cerebras puts disaggregated inference into production
Start here
Cerebras launched disaggregated inference, now in production, in which GPUs handle prompt processing (prefill) while Cerebras systems handle output generation (decode). In a measured production deployment on the Cerebras Inference Cloud, AMD Instinct GPUs processed prompts and CS-3 systems generated responses, delivering 5x the throughput per megawatt of a CS-3-only configuration with no loss in token generation speed. The company says it plans to deploy the approach across hundreds of megawatts in its inference cloud and partner deployments by the end of 2027, and offers partners an end-to-end software stack or a decode-only SDK; early-access partners include Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute.
From Hacker News · AI
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AICerebras launches disaggregated inference solution, with GPUs handling prefill and wafer-scale engines handling decode, delivering 5x throughput per megawatt
Cerebras launched disaggregated inference, now in production, in which GPUs handle prompt processing (prefill) while Cerebras systems handle output generation (decode). In a measured production deployment on the Cerebras Inference Cloud, AMD Instinct GPUs processed prompts and CS-3 systems generated responses, delivering 5x the throughput per megawatt of a CS-3-only configuration with no loss in token generation speed. The company says it plans to deploy the approach across hundreds of megawatts in its inference cloud and partner deployments by the end of 2027, and offers partners an end-to-end software stack or a decode-only SDK; early-access partners include Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute.
Heat for this event
Not enough continuous observations yet to draw a trend.