Terminal-Bench 4.1 released with hardened verifiers
Start here
Terminal-Bench released version 4.1 of its agent benchmark with improved verifiers designed to resist reward hacking. Re-grading TB 4.0 with the new verifiers required $0 in additional inference because it used saved artifacts; only about 1.16% of 11,550 trials changed score, and all 35 agents' updated scores stayed within their original TB 4.0 95% confidence intervals. Where regrade results were unavailable, the leaderboard keeps the original TB 4.0 score, so the update leaves the existing rankings essentially unchanged.
Generated from reports · updated 12 h ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AITerminal-Bench 4.1 Released with Hardened Verifiers Against Reward Hacking
Terminal-Bench 4.1 is released with improved verifiers designed to resist reward hacking, and regrading TB 4.0 with the new verifiers required $0 in additional inference cost. Only about 1.16% of 11,550 trials changed score, and all 35 agents' updated scores remain within their original TB 4.0 95% confidence intervals.
Heat for this event
Not enough continuous observations yet to draw a trend.