Skip to content
Hot eventDeveloping

Cerebras puts disaggregated inference into production

1 reports1 reporting sourcesUpdated 49 min ago

Start here

Report summary

Cerebras launched disaggregated inference, now in production, in which GPUs handle prompt processing (prefill) while Cerebras systems handle output generation (decode). In a measured production deployment on the Cerebras Inference Cloud, AMD Instinct GPUs processed prompts and CS-3 systems generated responses, delivering 5x the throughput per megawatt of a CS-3-only configuration with no loss in token generation speed. The company says it plans to deploy the approach across hundreds of megawatts in its inference cloud and partner deployments by the end of 2027, and offers partners an end-to-end software stack or a decode-only SDK; early-access partners include Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute.

From Hacker News · AI

Report timeline

Follow the reports to see the event from different sides.

November 18
  1. Hacker News · AI
    Cerebras launches disaggregated inference solution, with GPUs handling prefill and wafer-scale engines handling decode, delivering 5x throughput per megawatt

    Cerebras launched disaggregated inference, now in production, in which GPUs handle prompt processing (prefill) while Cerebras systems handle output generation (decode). In a measured production deployment on the Cerebras Inference Cloud, AMD Instinct GPUs processed prompts and CS-3 systems generated responses, delivering 5x the throughput per megawatt of a CS-3-only configuration with no loss in token generation speed. The company says it plans to deploy the approach across hundreds of megawatts in its inference cloud and partner deployments by the end of 2027, and offers partners an end-to-end software stack or a decode-only SDK; early-access partners include Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute.

Heat for this event

Not enough continuous observations yet to draw a trend.