Portal2Bench benchmark tests AI agents on Portal 2 chambers

Portal2Bench, an independent benchmark not affiliated with Valve, has published results measuring whether AI agents can solve 26 Portal 2 test chambers from chapters 2 to 5. A chamber counts as solved only if the agent reaches the next map within 300 plans and $30, with each chamber run fresh and independently and runs priced at API list prices. Run through Codex, GPT-6.1 Sol and GPT-6 Astra each solved 20 of 26 chambers, GPT-6.1 Terra solved 19, Opus 5.5 solved 11, Sonnet 5.5 solved 6, and GPT-6 Luna solved 0. Agents send key presses and view turns and receive screenshots and their position, with access only to a notes folder and no shell, web or console. Median cost per solved chamber ranged from $0.74 for GPT-6.1 Terra to $13.12 for Fable 5.1. Portal2Bench says most models run at max effort, with Gemini 4 Argon at high, and that most chambers are still being run because of compute constraints and usage limits.

Generated from reports · updated 13 h ago

Not enough continuous observations yet to draw a trend.