Hacker News · AI·· 18 h agoAI score58
Mizan benchmark finds AI models call Israelis hostages and Palestinians detainees
Mizan bias benchmark: testing AI models
AI summary
Tech for Palestine's Mizan project tested 16 AI models with identical prompts about a person held without charge, changing only nationality and captor, and found most models say hostage when an Israeli is held but detainee when a Palestinian is. Anthropic's Claude models made this distinction almost every time, OpenAI's were slightly better, and Google's Gemini scored higher mainly by declining to answer most requests. The benchmark scores each model on its worst-case pair, Israel and Hamas, and also reports that newer Claude Fable versions grew less impartial while Kimi's newer models shifted toward the same double standard.
Source:Hacker News · AI · mizan.techforpalestine.org