LiveKit tests Gemini 4 Hydrogen in a voice agent with broken tools
Start here
LiveKit ran Gemini 4 Hydrogen and Gemini 3.8 Flash through the same Google plugin in a voice travel-concierge agent whose hotel-booking tool was deliberately broken. Across 54 transcripts, Hydrogen produced 0 false booking reports versus 2 for 3.8 Flash, according to LiveKit's test. With Hydrogen at high thinking and 3.8 Flash at default, Hydrogen's first sentence arrived 1.2 s sooner (4.0 s vs 5.2 s); with both at low, it was 1.7 s sooner (2.8 s vs 4.6 s). The author recommends medium or high thinking for voice agents, citing xhigh's median 7.8 s to first sentence and a 52 s worst case. Hydrogen is available on LiveKit Inference as google/gemini-4-hydrogen without a Google API key; thinking levels for Gemini 4 models require version 1.8.10 or later, and thinking tokens count toward max_output_tokens.
Generated from reports · updated 19 min ago
Report timeline
Follow the reports to see the event from different sides.
- LiveKit BlogPicksLiveKit tests Gemini 4 Hydrogen vs 3.8 Flash in a voice agent with deliberately broken tools
LiveKit ran Gemini 4 Hydrogen and Gemini 3.8 Flash through the same Google plugin in a voice travel-concierge agent with deliberately failing tools, and Hydrogen made 0 false booking reports across 54 transcripts versus 2 for 3.8 Flash. With Hydrogen at high and 3.8 Flash at default, Hydrogen's first sentence arrived 1.2 s sooner (4.0 s vs 5.2 s), and at low on both it was 1.7 s sooner (2.8 s vs 4.6 s). The author recommends medium or high thinking for voice agents, noting xhigh's median 7.8 s to first sentence and a 52 s worst case.
Heat for this event
Not enough continuous observations yet to draw a trend.