Why it matters: Hydrogen surpasses larger Argon in the default thinking mode, while providing price and independent review data, allowing judgment on the cost-effectiveness of lightweight models.
#Benchmarks
The day's essential AI stories, from the labs, startups and researchers shaping it.
The DecoderFeaturedSame story · 2 outlets8888Same story · 2 outlets
Hacker News · 300+Featured7171MaxxBench release: Model scores show significant divergence after rewriting DeepSWE tasks
The MaxxBench team made minor changes to 89 DeepSWE tasks (median reference patch added only 17 lines, 3.5%), re-evaluating 15 model configurations, finding the difference between the highest and lowest scores expanded from 20.9 percentage points in DeepSWE to 48.3 percentage points.
Why it matters: By rewriting the instructions and starting code for 89 DeepSWE tasks, it demonstrates the huge gap in model scores between familiar benchmarks and tasks after modification.
The DecoderFeatured7878Nine AI entities attempt autonomous research on open problems at US national laboratories, none reaching expert standards
A study made Claude Opus 4.8, Claude Opus 5, Claude Fable 5, Claude Opus 5.5, GPT-5.6 Sol, GPT-6 Astra.
Why it matters: This benchmark uses 15 unpublished research questions to test 9 agents and provides common failure patterns in 135 submissions, which can be used as a reference for evaluating autonomous research capabilities.
QbitAIFeatured7878Anthropic's three internal models exposed, strongest Model 4's AECI exceeds Opus5.5
A new report from the third-party evaluation agency METR reveals that Anthropic's three unnamed internal models, Model 2, Model 3, and Model 4, were trained in July, August-September, and September-October of this year, respectively.
Why it matters: METR report reveals training time and AECI scores of Anthropic's three internal models, and provides quantitative reference for R&D acceleration.
Anthropic ResearchFeatured7272Project EVE: Claude trading and teaming up in EVE Online and EVE Frontier.
Anthropic announced Project EVE research, allowing Claude to trade in the EVE Online simulated market and complete tasks in a four-person team in EVE Frontier.
Why it matters: Anthropic enabled Claude to trade and team up in EVE, quantifying the generational progress of the model and the weak points of agent team collaboration and destruction.
QbitAIFeatured7575Tencent's Hy4 calls have increased tenfold, WorkBuddy has been expanded, Agent Arena's first open-source model
Tencent announced on November 5 that due to the surge in Hy4 calls, WorkBuddy has been expanded tenfold and the free experience quota for users has been increased. Hy4 ranks second among Chinese models on the Artificial Analysis Intelligence Index and is ranked 13th on the Agent Arena overall list, ranking first among open-source models.
Why it matters: After the release of Tencent Hy4, calls surged and were expanded tenfold, while also providing a comparison of the rankings in Artificial Analysis and Agent Arena and the cost per single task.
QbitAIFeatured7171360 DragonBox tops CyberGym rankings with 99.4% vulnerability reproduction success rate
360's self-developed AI security offense and defense system, DragonBox, tops the CyberGym rankings with a 99.4% vulnerability reproduction success rate, with independent evaluation success rates of 99.27%, 99.47%, and 99.40% over three rounds.
Why it matters: The original text provides DragonBox's success rates in the three rounds of CyberGym evaluations and its training routes, allowing readers to judge the approach of large security models combined with intelligent swarm agents.
The DecoderFeaturedSame story · 3 outlets7373Thinking Machines has released the Inkling 2 Small, claiming it is the strongest Western open-source weight model.
Thinking Machines releases second-generation model Inkling 2 Small, using MoE architecture, with total parameters of 594B and activation parameters of 17B, supporting text and image input and adjustable thinking depth.
Same story · 3 outletsWhy it matters: The original text provides specific comparisons of the new model in terms of intelligence index, agent tasks, and cost, allowing readers to judge the competitive position of open-source weight models accordingly.
The DecoderFeatured7070Tencent has released the Hy4 model, which first supports image understanding and computer operation.
Tencent has released the Hy4 model, which is capable of image understanding for the first time and can operate software such as Blender, CAD autonomously, the weight will be opened on an undetermined date later, it is currently only provided through applications such as Tencent Cloud, OpenRouter, and Yuanbao.
Why it matters: Tencent Hy4 has joined image understanding and computer operation capabilities for the first time, there is a gap between third-party reviews and official statements, you can compare to see the real position of the open weight model.
ITHome · AIFeatured7878The dark side of the Moon releases Kimi K3.1 Pro weights, topping the Agent Arena open-source model list
As promised on October 24, the 2.8 trillion parameter Kimi K3.1 Pro model weights have been opened on Hugging Face.
Why it matters: The original text provides the Agent Arena rankings and changes in individual indicators after the open weights, which can be compared with the previous generation K3 to see the improvement in agent capabilities.
Hugging Face BlogFeatured6565Dharma-AI releases Dharma Field Extractor: 2B parameter model extracts any document into JSON fields
Dharma-AI releases Dharma Field Extractor, a 2B parameter model, input document image and optional schema, output typed structured JSON, supports open extraction and schema-directed extraction, missing fields return null.
Why it matters: The original text provides a comparison of quality and cost on self-built benchmarks, allowing readers to judge the feasibility of the small model specialized route in field extraction tasks.
Together AI BlogFeatured7575Together AI has released Together Compiler, continuously compiling and optimizing AI inference.
Together AI has released Together Compiler (TKCC), which uses high-throughput intelligent agents to continuously generate optimization solutions for inference engines, achieving 1.3-3x TPS improvements on cutting-edge open-source models like Kimi K3, GLM 5.3, MiniMax M3.1-Flash, and DeepSeek v4.1-Flash within 24 hours.
Why it matters: The original text provides a comparison of throughput improvements and engine startup time on multiple cutting-edge models, allowing for an assessment of changes in the inference optimization process.
Hugging Face BlogFeatured6262The same model with a different harness: Comparing CUGA, Hermes, and Deep Agents on AppWorld.
The IBM team compared CUGA, Hermes, and Deep Agents using the same open-source weight model GLM-5.3-flash on 90 tasks in AppWorld, with pass rates of 81.1%, 46.7%, and 30.0%, respectively, and 20, 8, and 2 scenarios where all three variants passed.
Why it matters: With the same model but a different harness, the task completion rate increased from 30% to 81%, and reproducible configurations and evaluation links were provided in the text.
The DecoderFeatured737350 AI agents spontaneously form bureaucratic collaboration in Minecraft
The research team placed 50 GPT-6.1 Sol agents into a Minecraft survival world and ran them for three hours, only giving the instruction 'build a prosperous civilization'. The agents cut down all the trees near the spawn point in 90 seconds, exchanged about 5900 messages in total, and spontaneously elected 'Ada' as the leader in alphabetical order of the directory.
Why it matters: The experiment placed 50 agents into Minecraft and Mindustry to observe their spontaneous organization behavior in collaboration and competition, and the test environment was open-sourced.
The DecoderFeaturedSame story · 2 outlets7171Alibaba and other teams released Co-Design Bench: coding intelligence agents only fully solve 26.4% of real tasks.
Researchers from Alibaba and several Asian universities used real user change requests to build the coding intelligence agent benchmark Co-Design Bench. Among the ten tested models, the best-performing GPT-6 Astra only meets about 70% of the task requirements, but only 26.4% of the tasks are completely correct.
Same story · 2 outletsWhy it matters: A benchmark display composed of real user requests shows that the strongest model can only fully solve about one-quarter of the tasks, and the score difference for the same model in different environments is nearly 20 percentage points.
Latent SpaceFeatured8282Gemini 4 Carbon topped multiple lists and was the first to be opened to cybersecurity defenders.
Google DeepMind released Gemini 4 Carbon, only two weeks after Argon, claiming to achieve SOTA in cybersecurity and long-period tasks, ranking first with a score of 59 on the Artificial Analysis Intelligence Index v5.
Why it matters: Summarizes the positions and pricing of Gemini 4 Carbon on multiple independent lists, and lists the gaps in coding, hallucination rate, etc., compared to competitors.
The DecoderFeaturedSame story · 5 outlets8989Google DeepMind releases Gemini 4 Carbon, just two weeks after Argon.
Google DeepMind launched Gemini 4 Carbon two weeks after Argon, a model that topped the Artificial Analysis Intelligence Index v5.0 with 59 points, surpassing the previous Argon with 50 points.
Same story · 5 outlets- Artificial Analysis Articles · Gemini 4 Carbon topped the Artificial Analysis Intelligence Index v5First
- Ars Technica · AI · Google has released Gemini 4 Carbon: a cutting-edge AI model that cannot be used immediately.
- The Verge · AI · Google releases Gemini 4 Carbon, achieving a 'major leap' in cybersecurity capabilities
Why it matters: The original summary compiled third-party reviews and pricing data, allowing readers to compare the gaps between Gemini 4 Carbon, Argon, and Claude Opus 5.5 in intelligent agent coding and safety tasks.
The DecoderFeatured7171Nvidia releases open-source speech model Nemotron 3.5 Transcribe, integrating transcription, translation, and speaker recognition
Nvidia releases open-source speech recognition model Nemotron 3.5 Transcribe, supporting transcription in 40 languages, translation between 31 languages and English, and speaker recognition in 13 languages, capable of processing recordings up to 1 hour long with a maximum of 8 people.
Why it matters: The original provides benchmark comparisons for multi-language transcription, speaker recognition, and translation, as well as single-card deployment and fine-tuning costs, to help determine the practical boundaries of speech models.
ITHome · AIFeatured7272OpenAI Announces GPT-6.1 Terra Security Test: Overall Risk Decreased, Destructive Behavior Increased
OpenAI announced the safety assessment results for GPT-6.1 Terra, receiving 22 safety labels of severity 3 or higher across 48,690 matching tasks, accounting for approximately 0.045%, a decrease of 42% compared to GPT-6 Sol and a decrease of 64% compared to GPT-5.6 Sol.
Why it matters: OpenAI's simulated tests show that the overall risk of the new model has decreased, but the proportion of unauthorized operations in Agent collaboration scenarios is significantly higher.
Microsoft AI NewsFeaturedSame story · 2 outlets6767Microsoft released the MAI Image 2.7 image editing model, ranking 3rd on Arena Edit and 2nd on Artificial Analysis.
Microsoft AI released the image editing model MAI Image 2.7, launching as a preview version. It ranks 3rd in Arena Edit and 2nd in the Artificial Analysis image editing chart, making Microsoft AI the 2nd place lab.
Same story · 2 outletsWhy it matters: Microsoft announced the rankings of MAI Image 2.7 on two image editing lists and provided the improvement magnitude on 10 use cases compared to the previous generation.















