Skip to content
Live
Trending storyPast story

Java framework migration leaderboard launched

1 reports1 outlets22 days agoUpdate

The story so far

AI overview

IBM and Kaggle have launched the Java framework migration benchmark ScarfBench to evaluate the ability of AI agents to migrate enterprise Java applications. Among 204 migration tasks, Gemini 4 Argon paired with Antigravity ranked first (smoke test 75.2%), and Claude Opus 5.5 paired with Claude Code ranked second (100% pass rate 72.1%). Kaggle participated in the design, independently ran the benchmark, and sponsored computing resources to maintain the benchmark to support new model releases. Public model submissions are not available at launch but are planned.

AI-written from the reports · Updated 22 days ago

Coverage timeline

Follow the coverage to see every side of the story.

27 Oct
  1. IBM Research Blog
    IBM and Kaggle launch Java framework migration leaderboard

    IBM partnered with Kaggle to launch the Java framework migration benchmark ScarfBench, evaluating the ability of AI agents to migrate enterprise Java applications. Among 204 migration tasks, Gemini 4 Argon paired with Antigravity ranked first (smoke test 75.2%), and Claude Opus 5.5 paired with Claude Code ranked second (100% pass rate 72.1%).