Java framework migration leaderboard launched
The story so far
IBM and Kaggle have launched the Java framework migration benchmark ScarfBench to evaluate the ability of AI agents to migrate enterprise Java applications. Among 204 migration tasks, Gemini 4 Argon paired with Antigravity ranked first (smoke test 75.2%), and Claude Opus 5.5 paired with Claude Code ranked second (100% pass rate 72.1%). Kaggle participated in the design, independently ran the benchmark, and sponsored computing resources to maintain the benchmark to support new model releases. Public model submissions are not available at launch but are planned.
AI-written from the reports · Updated 22 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- IBM Research BlogIBM and Kaggle launch Java framework migration leaderboard
IBM partnered with Kaggle to launch the Java framework migration benchmark ScarfBench, evaluating the ability of AI agents to migrate enterprise Java applications. Among 204 migration tasks, Gemini 4 Argon paired with Antigravity ranked first (smoke test 75.2%), and Claude Opus 5.5 paired with Claude Code ranked second (100% pass rate 72.1%).