Standard Kernel uses AI to optimize Intel Gaudi2 past Nvidia H100
Start here
Standard Kernel says software optimization alone made Intel Gaudi2 faster than an Nvidia H100 80GB PCIe for long-context inference on Qwen3.5-27B. In its own benchmark, Gaudi2 reached 60% higher decode throughput and 8% higher cached-prefill throughput, while costing 17% as much per hour to rent. The company says its changes to vLLM and the vllm-gaudi plugin raised Gaudi2's cached-prefill throughput 3.21x and decode throughput 2.84x over baseline, producing 9.4x more output tokens per dollar. The test used 16 concurrent requests with 70,000-token inputs and a 95% shared prefix, so the result applies to that long-context, cache-heavy workload rather than to inference generally.
Generated from reports · updated 1 d ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AIStandard Kernel uses AI to optimize Intel Gaudi2 past Nvidia H100 on long-context inference
Standard Kernel reports that software optimization alone lifted Qwen3.5-27B inference on Intel Gaudi2 to 60% higher decode throughput and 8% higher cached-prefill throughput than an Nvidia H100 80GB PCIe, at 17% of the hourly rental cost. The company says its changes to vLLM and the vllm-gaudi plugin raised Gaudi2's cached-prefill throughput 3.21x and decode throughput 2.84x over baseline, yielding 9.4x more output tokens per dollar. The benchmark used 16 concurrent requests with 70,000-token inputs and a 95% shared prefix.
Heat for this event
- Current, comparable range
- 5
- Peak, comparable range
- 10November 19 02:00
- Change in 24 hours
- -50%
The trend compares the same participants over hours observed continuously, so its range can be narrower than the current heat count. Hover or tap the chart for hourly heat; the arrow keys move through it.