Benchmarks

Why it matters: A benchmark display composed of real user requests shows that the strongest model can only fully solve about one-quarter of the tasks, and the score difference for the same model in different environments is nearly 20 percentage points.

Why it matters: By rewriting the instructions and starting code for 89 DeepSWE tasks, it demonstrates the huge gap in model scores between familiar benchmarks and tasks after modification.

Why it matters: This benchmark uses 15 unpublished research questions to test 9 agents and provides common failure patterns in 135 submissions, which can be used as a reference for evaluating autonomous research capabilities.

Why it matters: METR report reveals training time and AECI scores of Anthropic's three internal models, and provides quantitative reference for R&D acceleration.

Why it matters: Anthropic enabled Claude to trade and team up in EVE, quantifying the generational progress of the model and the weak points of agent team collaboration and destruction.

Why it matters: The original text provides specific comparisons of the new model in terms of intelligence index, agent tasks, and cost, allowing readers to judge the competitive position of open-source weight models accordingly.

Why it matters: After the release of Tencent Hy4, calls surged and were expanded tenfold, while also providing a comparison of the rankings in Artificial Analysis and Agent Arena and the cost per single task.

Why it matters: The original text provides DragonBox's success rates in the three rounds of CyberGym evaluations and its training routes, allowing readers to judge the approach of large security models combined with intelligent swarm agents.

Why it matters: Tencent Hy4 has joined image understanding and computer operation capabilities for the first time, there is a gap between third-party reviews and official statements, you can compare to see the real position of the open weight model.

Why it matters: The original text provides the Agent Arena rankings and changes in individual indicators after the open weights, which can be compared with the previous generation K3 to see the improvement in agent capabilities.

Why it matters: The original text provides a comparison of quality and cost on self-built benchmarks, allowing readers to judge the feasibility of the small model specialized route in field extraction tasks.

Why it matters: The original text provides a comparison of throughput improvements and engine startup time on multiple cutting-edge models, allowing for an assessment of changes in the inference optimization process.

Why it matters: With the same model but a different harness, the task completion rate increased from 30% to 81%, and reproducible configurations and evaluation links were provided in the text.

Why it matters: The experiment placed 50 agents into Minecraft and Mindustry to observe their spontaneous organization behavior in collaboration and competition, and the test environment was open-sourced.

Why it matters: Summarizes the positions and pricing of Gemini 4 Carbon on multiple independent lists, and lists the gaps in coding, hallucination rate, etc., compared to competitors.

Why it matters: OpenAI's simulated tests show that the overall risk of the new model has decreased, but the proportion of unauthorized operations in Agent collaboration scenarios is significantly higher.

Why it matters: Microsoft announced the rankings of MAI Image 2.7 on two image editing lists and provided the improvement magnitude on 10 use cases compared to the previous generation.

Why it matters: The original text used the scores of the Intelligence Index and Coding Agent Index, along with the cost per task, to quantify the change in cost-effectiveness of GPT-6.1 Terra compared to GPT-5.6 Terra.

Why it matters: Research fixed the training data size while only increasing the number of applications, observing an improvement in success rates for unseen applications, providing a reproducible path for data construction of mobile agents.