Skip to content
Hot eventDeveloping

Sakana AI Releases JFinBench Japanese Financial Benchmark

1 reports1 reporting sourcesUpdated Just now

Start here

AI digest

Sakana AI released JFinBench, a Japanese financial benchmark built from six 2026 financial certification exams totaling 321 questions, including written and image-based items, and published the dataset, code and leaderboard. Across 12 LLMs, Claude Fable 5 scored 95.8 and GPT-6 Astra 93.6; the best open-weight model, Kimi K3.1 Pro, reached 83.4, about 10 points behind GPT-6 Astra despite reportedly matching it on coding benchmarks. Sakana AI says most failures came from missing knowledge of Japanese regulations rather than calculation errors. The benchmark was built with a human-in-the-loop agentic pipeline that took about 12 hours of human verification.

Generated from reports · updated 12 min ago

Report timeline

Follow the reports to see the event from different sides.

November 16
  1. Sakana AI Blog
    Sakana AI releases JFinBench, a Japanese financial benchmark built from 2026 certification exams

    Sakana AI released JFinBench, a Japanese financial benchmark built from six 2026 financial certification exams totaling 321 questions, including written and image-based items. Across 12 LLMs, Claude Fable 5 scored 95.8 and GPT-6 Astra 93.6, while the best open-weight model, Kimi K3.1 Pro, reached 83.4, about 10 points behind GPT-6 Astra despite reportedly matching it on coding benchmarks. Most failures came from missing knowledge of Japanese regulations rather than calculation errors, and the benchmark was built with a human-in-the-loop agentic pipeline that took about 12 hours of human verification.

Heat for this event

Not enough continuous observations yet to draw a trend.