Skip to content
Hacker News · AI·· 2 h agoAI score64

Harvard study finds Claude Fable ban barely changed developer merge rates

Downgrading from Claude Fable barely affected dev productivity during ban

AI summary

A Harvard study tracking 297 Claude Fable 5 users through the 19-day ban found that access to the model raised the share of pull requests merged within seven days by just 1.4 percentage points, a difference the paper calls not statistically significant. Across five Claude upgrades in 2026 covering 1,208 developers and more than 15,000 code submissions, no significant gain appeared in merge rates or weekly completed changes, even where benchmark scores rose 5 to 18 points. The authors suggest the tasks developers hand to agents already fall within what older models can do, so benchmark scores may not reflect real work productivity.

Source:Hacker News · AI · unite.ai