GitHub releases ReviewBench, an open benchmark for AI code review
GitHub released ReviewBench, an open benchmark for AI code review agents built on 219 public pull requests across 19 languages, with language and repository-size distributions modeled on 103.9 million GitHub pull requests. The golden set draws findings from human reviewers, author follow-up commits, static analysis and multiple frontier LLMs, graded by Claude Sonnet 5 under a published rubric, and senior engineers independently re-labeled every finding with 96.6% agreement. GitHub says its offline evaluation of Copilot code review anticipated the direction of a production A/B test, where addressed rate rose 8.0%, recall rose 13.6% and cost per review fell 8.0%.