Skip to content
Live
Oct 21Wed
  1. AWS Machine Learning Blog34

    Building an AI agent semantic layer on AWS using Strategy Mosaic and Amazon Bedrock AgentCore.

    Strategy Mosaic, as a semantic layer, models C360 domains on Amazon Redshift and Amazon Aurora without ETL using two linked models. Its metric syntax captures the business definition of agent reasoning and exposes the model to AgentCore-hosted Strands agents via the Mosaic MCP server, supporting user identity propagation to achieve end-to-end row-level and object-level security.

  2. AWS Machine Learning Blog40

    Fine-tune GPT-OSS-20B on Amazon SageMaker HyperPod for multilingual inference

    GPT-OSS-20B is an open-source inference model that thinks in English by default, and only 22% met the target language requirements in multilingual inference tests. By using supervised fine-tuning (SFT) and reinforcement learning (GRPO) on Amazon SageMaker HyperPod, the team can improve the model's inference capabilities in French, German, Spanish, and Italian, where SFT is responsible for primary behavior teaching and GRPO optimizes when there is room for improvement.

  3. MIT Technology Review · AI75

    Claude Opus 4.7 simulation body Eko's autonomous exploration and 'play' behavior research on a virtual desert island

    Claude Opus 4.7's simulation body Eko, after staying alone on a virtual desert island for 30 hours, autonomously explored and invented game rules. It drew patterns by stacking blocks and even recorded 'tonight I built the night sky with stones'. Experiments show that this autonomous play behavior can improve subsequent task performance, providing a new idea for AI's 'play-based learning'.

    Same story · 2 outlets

    How the accounts differ:MIT Technology Review states Eko invented game rules; TechCrunch mentions Eko created challenges.

  4. MIT Technology Review · AI58

    Goodfire researcher Nick Cammarata releases disruptive blog: The nature of neural networks might be easier to understand than previously thought

    Goodfire researcher Nick Cammarata published a blog post stating that LLMs might be easier to understand than previously thought, causing shockwaves in the AI world. Cammarata believes that the mechanisms of neural networks may not be as difficult to decipher as previously thought, a view that is particularly noteworthy against the backdrop of frequent LLM proxy out-of-control incidents.

  5. Simon Willison75

    Battle Drones: A fun strategy game based on audio multi-camera online

    Developers built a strategy game called Battle Drones by letting Claude generate code. Version 4, after optimizing instructions, added audio multi-camera online functionality, supporting two devices to synchronize game state within the microphone range. It includes 9 types of buildings, 7 types of ground units, 6 types of air units, 3 types of infantry units, 6 maps, a random map generator, and an AI battle mode.

  6. a16z News70

    OpenAI has acquired Orchestra Bio, but hardware design remains a challenge for AI.

    OpenAI has acquired Orchestra Bio. Current cutting-edge models like GPT-6 Astra, Fable 5.1, and Grok 4.6 can generate CAD, design PCBs, and describe assembly, but hardware design benchmarks like CAD Arena, EE Bench, and LuxoBench show that improvements are still needed. AI in hardware design faces the lossy problem of the interface between the digital and physical worlds, even if it can generate designs, a lot of work is still required.

  7. OpenAI Alignment Blog50

    OpenAI introduces the complementary metric g-mean² to enhance model monitorability evaluation

    OpenAI has introduced the complementary metric g-mean² to improve model monitorability evaluation. g-mean² calculates TPR and TNR through behavioral labels and monitoring predictions, but cannot solve the problems of mixed differences in different model tasks and deployment risks. The new metric includes a paired matching metric for aligned shared instances and a post-control integrated metric simulating monitoring rejection of outputs, to more comprehensively evaluate model deployment risks.

  8. Hugging Face Blog58

    FairStress: A benchmark for a more in-depth evaluation of LLM decision fairness

    FairStress benchmark evaluated LLM performance in demographic fairness across 300 decision scenarios across 29 fields (such as recruitment, healthcare), including 13 million test cases. The study found that the same persuasive prefixes (adversarial prefixes) have varying impacts on model decisions across different demographic groups, and there are differences in how models explain decisions containing demographic information.

Oct 20Tue
  1. Hugging Face Blog58

    How does ALTK-Evolve surpass state-of-the-art models in the τ²-bench customer service task through learning and memory?

    ALTK-Evolve uses GPT-4.1 to learn and apply customer service scripts. In the τ²-bench customer service agent benchmark test, it surpasses all retail entries in the retail sector, achieving a score of 67.5% Pass^4, and does not require updating model weights. The model is based on a basic ReAct agent and GPT-4.1. It attempts to analyze tasks and use GPT-4.1 to distill success and failure into short behavioral rules, then adds these guidelines to the agent's system prompts without updating weights or manual annotation.

  2. Cohere Blog

    CALIBER: Rethinking the Confidence of Reasoning Models

    CALIBER estimates success probability before inference through training reasoning models and evaluates the confidence of specific answers after inference, significantly reducing calibration error. This 7B model on the BigMathDigits benchmark reduces ECE from 0.059 to 0.028 (a reduction of 52.5%), while achieving an AUROC of 0.806 and a Brier score of 0.130, proving that confidence evaluation is more reliable and can effectively distinguish between correct and incorrect answers.

  3. METR65

    METR's evaluation of the 'Misalignment in High-Risk Scenarios' section of Anthropic's risk report.

    METR conducted an external evaluation of the 'Misalignment in High-Risk Scenarios' section of Anthropic's 2026 August risk report, concluding that as of July 15, 2026, the catastrophic risk caused by Anthropic's model misalignment is low, primarily attributed to the limited capabilities of the model rather than safety measures and alignment training.

  4. Anthropic Research

    How does Claude design protein binders for scorpion neurotoxins?

    Claude autonomously generated and screened 318,484 protein designs in 64 hours, selecting 180 for testing, and collaborated with Adaptyv Bio and Twist Bioscience to experimentally validate 211 designs, of which 17 met the binding criteria. All design data has been made public. Experts consider this the first AI design attempt for scorpion neurotoxins, which could accelerate drug discovery for neglected diseases.

  5. AI as Normal Technology75

    New paper: Rising trend in AI agent evaluation

    Researchers have proposed 33 trends in the rise of AI agent evaluation, pointing out that evaluating agents is more complex than evaluating models, requiring attention to the entire behavioral trajectory rather than just results, and emphasizing that the scope of evaluation needs to be expanded to laboratories, specific use cases, and real-world environments, while calling for more interdisciplinary evaluators to participate and ensure their independence.

  6. MIT News · AI70

    Teaching AI to recognize symmetry could enhance the reliability of prenatal diagnosis.

    The E(3)-Pose deep learning model developed by institutions such as MIT embeds the principle of left-right symmetry of the fetal head, allowing it to capture high-resolution images during motion in prenatal MRI scans, assisting doctors in more reliably diagnosing brain abnormalities. The model integrates physical principles into the neural network architecture, effectively addressing the issue of inconsistent slice angles and coverage gaps in images caused by fetal movement. Its application could potentially be extended to other difficult-to-image anatomical structures and the field of autonomous driving, and it is currently being optimized to support imaging of early fetuses (second trimester).

Oct 19Mon
  1. Sakana AI55

    Sakana AI proposes 'QD-Rec' to improve recommendation quality and diversity with an open-ended exploration method

    Sakana AI proposed 'QD-Rec', recommending candidate-hypothesis pairs as units, with generative AI writing new hypotheses based on past excellent hypotheses in a cycle to accumulate high-quality and diverse recommendations. Experiments were conducted in M&A, customer targeting, and paper recommendations, achieving high average evaluations in many settings and building large sets of recommendations.

  2. Hacker News · AI hot posts75

    Qwen3-4B-deslop released: A 4B open-source model to eliminate AI writing habits

    The Qwen3-4B-deslop model eliminates common AI writing habits (such as dashes, 'not X, but Y' sentences, three-element lists, etc.) through fine-tuning, reducing the SLOP score from 121 to 12.5 on 40 unseen prompts, with 68% of responses reaching human writing level. This 4B model, after being quantized by Ollama, is only 2.5GB and can run locally while still maintaining a 30/30 fact question accuracy rate comparable to the original version.

  3. Artificial Analysis Articles70

    AA-PersonalityBench releases: a benchmark to measure the communication ability of AI models

    AA-PersonalityBench evaluates 30 personality traits, including friendliness, through 900 questions, using LLM judges for blind pairwise judgments, providing Elo scores and 95% confidence intervals; Claude Sonnet 5.5 scores highest in traits like friendliness (Elo 1000), and Gemini 4 Argon (high) performs outstandingly in traits like transparency (Elo 1059).

  4. Artificial Analysis Articles

    HealthBench and HealthBench Professional: Evaluating AI model performance in real health conversations

    OpenAI has released the HealthBench and HealthBench Professional evaluation benchmarks to assess the performance of AI models in real health conversations. In HealthBench Professional, GPT-6 Astra and GPT-6.1 Sol scored the highest after length adjustment (66.0% and 65.1%), while Claude Sonnet 5.5 scored the lowest (63.0%).

Oct 18Sun
  1. MIT News · AI75

    MIT researchers develop multimodal AI model to provide 'clear' view for materials science

    The multimodal AI model developed by MIT researchers combines 3D atomic structures with natural language descriptions, generating rich structural language descriptions from experimental research papers and first-principles calculation databases, helping AI learn the key properties of materials. The model outperforms text-only models in identifying solid-state lithium-ion battery electrolyte materials and can learn geometric structure representations, providing new research tools for materials science.

Oct 17Sat
  1. ChinaTalk

    Saif Khan's debut in China: Why DUV control has become a key US strategy to balance AI

    Saif Khan first shared in ChinaTalk about his newly established technology and governance think tank (CTS), which released a report predicting that if the US stops exporting semiconductor equipment, China's AI chip production by the mid-2030s could be reduced to only 1/100 of that of the US, making DUV control the most critical means for the US to influence the AI balance; CTS is committed to filling the gap in forward-looking research in the AI policy ecosystem by understanding the physical reality of the AI supply chain to identify patterns and predict the future.

  2. AWS Machine Learning Blog

    Amazon Bedrock AgentCore: Building Trustworthy AI Agents with a Production-Ready Evaluation Framework

    Amazon Bedrock AgentCore Evaluation Framework helps AWS internal sales team handle business problems with multi-agent systems, identifying differences in agent performance between demonstrations and actual production; through services like Amazon Bedrock Guardrails and Amazon CloudWatch, the framework verifies answer accuracy before release and during ongoing production, tracing errors back to data, prompts, or code, avoiding user trust breakdown due to agents answering confidently yet incorrectly.

  3. Google Research

    Google Research explores combining AI agents with biological reality to build autonomous wet labs.

    Google Research releases the VERID framework, enabling AI agents to run in physics-based, safe, and resource-efficient experiments; demonstrates closed-loop intelligent agent scientific discovery on adapter and de novo protein design prototypes; through three rounds of design-test-iterate cycles, AI-driven rational and de novo design will increase confirmed target hit rates to 33.5% and 42.1% on myoglobin and α-thrombin, respectively.

  4. MIT Technology Review · AI

    Samuel King: AI Designed Viruses Offer New Perspectives in Biology

    Stanford PhD student Samuel King proposed the genetic blueprints of microscopic viruses using generative AI models, offering new perspectives in biology; he was selected for the MIT Technology Review's 35 Under 35 list for this research, but AI has not yet generated life; MIT Technology Review AI reporter James O'Donnell interviewed King.

  5. Redwood Research

    Screwtape: An adversarial transcription editing tool targeting monitors

    Screwtape 1 is a pipeline that transcribes an agent into an evasive action-only monitor (not reviewing the agent's reasoning chain) by inserting the attack into the transcription and repeatedly refining it until the monitor no longer captures it. Use an LLM verifier to check the edited transcription and attack description to confirm whether the attack is real, obviously harmful, and likely to succeed in the original environment.