Sakana AI Blog· Sakana AI·· Just nowAI score58
Sakana AI releases JFinBench, a Japanese financial benchmark built from 2026 certification exams
JFinBench: 最新の金融資格試験を用いた日本語金融ベンチマークの公開
AI summary
Sakana AI released JFinBench, a Japanese financial benchmark built from six 2026 financial certification exams totaling 321 questions, including written and image-based items. Across 12 LLMs, Claude Fable 5 scored 95.8 and GPT-6 Astra 93.6, while the best open-weight model, Kimi K3.1 Pro, reached 83.4, about 10 points behind GPT-6 Astra despite reportedly matching it on coding benchmarks. Most failures came from missing knowledge of Japanese regulations rather than calculation errors, and the benchmark was built with a human-in-the-loop agentic pipeline that took about 12 hours of human verification.
Source:Sakana AI Blog · sakana.ai