Skip to content
Hot eventWatching

Standard Kernel uses AI to optimize Intel Gaudi2 past Nvidia H100

1 reports1 reporting sourcesUpdated 1 d ago

Start here

AI digest

Standard Kernel says software optimization alone made Intel Gaudi2 faster than an Nvidia H100 80GB PCIe for long-context inference on Qwen3.5-27B. In its own benchmark, Gaudi2 reached 60% higher decode throughput and 8% higher cached-prefill throughput, while costing 17% as much per hour to rent. The company says its changes to vLLM and the vllm-gaudi plugin raised Gaudi2's cached-prefill throughput 3.21x and decode throughput 2.84x over baseline, producing 9.4x more output tokens per dollar. The test used 16 concurrent requests with 70,000-token inputs and a 95% shared prefix, so the result applies to that long-context, cache-heavy workload rather than to inference generally.

Generated from reports · updated 1 d ago

Report timeline

Follow the reports to see the event from different sides.

November 19
  1. Hacker News · AI
    Standard Kernel uses AI to optimize Intel Gaudi2 past Nvidia H100 on long-context inference

    Standard Kernel reports that software optimization alone lifted Qwen3.5-27B inference on Intel Gaudi2 to 60% higher decode throughput and 8% higher cached-prefill throughput than an Nvidia H100 80GB PCIe, at 17% of the hourly rental cost. The company says its changes to vLLM and the vllm-gaudi plugin raised Gaudi2's cached-prefill throughput 3.21x and decode throughput 2.84x over baseline, yielding 9.4x more output tokens per dollar. The benchmark used 16 concurrent requests with 70,000-token inputs and a 95% shared prefix.

Heat for this event

Current, comparable range
5
Peak, comparable range
10November 19 02:00
Change in 24 hours
-50%

The trend compares the same participants over hours observed continuously, so its range can be narrower than the current heat count. Hover or tap the chart for hourly heat; the arrow keys move through it.