Sakana AI· Sakana AI·Added· 16 hours ago58
Sakana AI has developed the Japanese financial benchmark JFinBench and released related resources
Brief overview
Sakana AI has released the Japanese financial benchmark JFinBench based on the 2026 version of Japan's six major financial qualification exams, evaluating the performance of 12 LLMs and finding that major open-weight models like Kimi K3.1 Pro have a performance gap of about 10 points compared to proprietary models like GPT-6 Astra on JFinBench, primarily due to insufficient knowledge of Japanese institutional knowledge. The benchmark includes 321 questions, covering multiple-choice, fill-in-the-blank, essay, and image questions, and adopts a human-machine collaborative automated construction process that can be updated quickly.
Source: Sakana AI · sakana.ai