Skip to content
Live
#Safety & alignment
Oct 9Fri
  1. TechCrunch · AI65

    Anthropic updates use policy to prohibit model abuse and election interference

    Anthropic updates use policy, adding prohibitions on election interference, weapon software, and surveillance, and explicitly prohibits long-term verbal abuse of the Claude model. This policy update only applies to cases where users repeatedly maliciously treat the model, and does not involve normal frustration, creative themes, or model testing. The policy also prohibits using Claude to run false accounts or fake news, and to deceive voters or disrupt elections.

  2. TechCrunch · AI70

    OpenAI's mathematical solutions have not yet reached the standards of the field

    OpenAI released hundreds of solutions this week claiming to solve world-class mathematical problems, but failed to meet mathematical standards, especially in requiring human understanding of mathematical results. A new paper points out that there is a gap between the natural language solution and the formal expression solution of one of the million-dollar problems solved by OpenAI models.AGMAI points out that OpenAI only partially followed its recommendations, for example, only 10 papers publicly disclosed the model reasoning chain, 42% of the proofs were not formally processed, and did not include machine-readable metadata linking natural language to formal results.

Oct 8Thu
  1. The Verge · AI65

    Meta Muse and OpenAI Dots: Competition in the landscape of next-generation consumer AI agents

    Meta Muse and OpenAI Dots are both AI model-based proxy platforms designed to provide personal assistant services to users. Muse is free to use, while Dots is paid. Both can handle tasks ranging from simple ones (such as restaurant reservations and email filtering) to more complex ones (such as marketing, legal work, and accounting).

  2. The Decoder75

    Two Ethereum researchers warn that AI-assisted mathematics may crack encryption wallet signature systems within months.

    Ethereum researchers Justin Drake and Vitalik Buterin warn that AI-assisted mathematics could break the ECDSA signature system for encryption wallets in the worst case within months. Drake suggests transferring funds to addresses that have never signed transactions to protect private keys, while Buterin advises against acting too quickly, advocating for long-term reliance on architectures based solely on hash functions, as even quantum-safe lattice schemes could be weakened by AI advancements. Currently, no one has actually cracked the current ECDSA signature scheme.

  5. The Decoder70

    The ARTEX tool, utilizing AI, launched an attack that targeted several banks in South Korea.

    ARTEX, an open-source penetration testing tool using DeepSeek v4.1-flash, GLM-5.3, and Grok 4.6 AI models, was suspected of being used by Chinese attackers to attack multiple South Korean financial institutions from late September to early October 2026, stealing large amounts of data, including over 25,000 records from Shinhan Bank. This incident highlights the risk of AI tools exacerbating the threat of large-scale single-point cyber attacks.

  6. MIT Technology Review · AI

    AVEVA explores the path of security autonomization in industrial AI.

    AVEVA's Chief Technology Officer Arti Garg notes that industrial AI is entering a new phase, with advancements in foundational models, physical AI, and agent technologies enabling it to automate more complex tasks. However, security, reliability, and infrastructure risks need to be addressed. Industrial AI needs to integrate multi-source data, allow AI to assist rather than replace human decision-making, and establish governance frameworks to ensure safety and efficiency. Organizations need to redesign business processes and equip experienced workers with new skills to achieve safer, more efficient, and sustainable automation.

  9. Artificial Analysis Articles75

    Artificial analysis of network indices introduces a controlled access model

    Artificial analysis of the network index includes GPT-6 Sol (Dawn Blue, largest version) in the evaluation, the model accessed through OpenAI's Dawn program ranked first in the index, with an overall score 32 points higher than the public version of GPT-6 Sol (largest version) and no safety interception, task cost is $1.77 per item, better than Grok 4.7 ($11.67 per item), and performed the best in the CyberGym-E2E benchmark test. The index will continue to expand to cover publicly and controlled access models.

  10. Anthropic Research

    Anthropic has launched an open-source software vulnerability discovery service.

    Anthropic has launched an optional vulnerability discovery service, OSS Scanner, which uses large models like Claude Mythos to scan open-source code, providing free security audits, but outputs are not manually reviewed. The service has disclosed 97 high-risk vulnerabilities to 48 projects, of which 88% meet CVD standards, receiving positive feedback from projects such as PostgreSQL and OpenSSL.

  11. AWS Machine Learning Blog

    How Amazon Quick and Amazon Bedrock Knowledge Bases perform RAG access control through real-time ACLs

    Amazon Quick and Amazon Bedrock Knowledge Bases address the RAG access control challenge through real-time access control list (ACL) access, verifying permissions directly with authoritative sources during queries, avoiding reliance on potentially outdated or incorrectly mapped ACL data. The system employs a two-phase architecture: the first phase performs pre-retrieval filtering, and the second phase calls the data source API in real-time to validate candidate documents, striking a balance between performance and security, ensuring AI responses always comply with the latest permission requirements.

  12. Anthropic Newsroom

    Anthropic has launched a cybersecurity mission program focused on critical infrastructure and open-source software security

    Anthropic announced its cybersecurity mission plan, supporting defenders through cutting-edge models, on-site engineers, and threat research. The plan initially focuses on critical infrastructure (launching the Critical Infrastructure Defense Program CIDP) and open-source software (launching the OSS Scanner to provide free security scanning), with core partners including Accenture, Booz Allen, etc. The plan aims to address the risk of AI being maliciously used and alleviate the resource shortage problem for defenders.

Oct 7Wed
  1. Platformer65

    Anthropic and other companies discuss limiting the intelligence ceiling of large language models

    Anthropic CEO Dario Amodei called for limits on recursive self-improvement. Anthropic is responsible for extending policies that restrict the training and deployment of more powerful systems. Participants may want to limit the level of intelligence future systems are allowed to reach, or implement it by restricting the use of cutting-edge models for AI research, computational resources, or model deployment capabilities, but the execution capability is unknown.

  2. Epoch AI35

    The probability of U.S. adults encountering a cyber event after the release of Claude Fable 5 remains unchanged.

    The proportion of US adults reporting at least one cyber event in the past 12 months did not show a reliable change after the release of Claude Fable 5, decreasing from 46% (90% CI: 44% to 49%) in June to 45% (42% to 47%) in September, and there was no significant difference in the proportion of respondents frequently using AI (from 53% to 51%).

  4. Ars Technica · AI65

    OpenAI will default to watermarking ChatGPT for the EU

    OpenAI will default to adding watermarks to text generated by ChatGPT in the European Union region, aiming to comply with the EU AI regulation requirements that took effect in August. The EU-specific watermark technology is called textGrain, which uses proprietary algorithms to embed specific patterns into word selection that are difficult for humans to detect, requiring a dedicated detector to identify, but OpenAI acknowledges that it may be circumvented by individuals with basic technical skills. The company states it will provide detectors to a limited number of researchers and institutions and will establish an application process for other institutions to join.

  5. Microsoft Research

    Microsoft Research host Chad Atalla interviews Jennifer Neville: Lessons from AI's Errors and Failures

    Microsoft Research host Chad Atalla discusses with Jennifer Neville the role of AI evaluation in pushing the boundaries of AI system performance, and the 'surprising failures' that occur when models exceed traditional benchmark tests. Jennifer Neville shares practical guidance for working with current AI systems and discusses the importance of carefully reviewing data when results do not match expectations, as well as the insights she has gained from decades of AI progress in predicting future trends.

Oct 6Tue
  4. Google Research

    Google Research explores the open and emergent challenges in agent privacy and security from a contextual perspective.

    Google Research has released a new report titled 'Privacy and Security Challenges of Open and Emergent Agents: A Contextual Perspective,' analyzing the fundamental privacy and security challenges faced by autonomous agents, requiring coordinated defense at the system, model, user, and ecosystem levels. The report indicates that agents need to handle personal data and act autonomously, but their characteristics in unstructured interfaces, probabilistic control flow, and autonomous agency introduce new challenges, requiring strategies that combine contextual integrity theory to evaluate the social and contextual appropriateness of actions.

Oct 5Mon
  1. Cohere Blog

    Cohere Northbound 2.0: A no-compromise enterprise-grade AI platform

    Cohere releases Northbound 2.0 platform, ending the compromise in deploying AI agents for enterprises in terms of security, intelligence, cost, and control. The platform supports knowledge internalization, reusable agents, and automated processes, featuring enterprise-grade secure deployment, permission management, real-time monitoring, and cost tracking functions, adopted by companies such as LG CNS and Bell Cyber.

Oct 3Sat
  1. Redwood Research70

    Capability research extends the safety-utility Pareto front

    Capability research typically extends the safety-utility Pareto frontier by increasing utility, forcing developers to sacrifice safety; conversely, safety research typically extends the frontier to enhance safety without significantly increasing utility. Developers choose specific points on the Pareto frontier based on its shape, tending to increase utility when risks rise, leading to safety research generally improving safety more than capability research.

  2. Big Technology70

    Anthropic's IPO and our shared leap of trust

    Anthropic's 2024 revenue grew to $4.6 billion, but its infrastructure spending last year was $7.3 billion, and it is expected to invest $518 billion in the next decade, of which about 80% are irreversible commitments. The company needs to maintain growth amid fierce competition, increasing political risks, and tighter funding, otherwise its massive financial obligations may trigger systemic risks. The AI industry is facing challenges of declining advantages of cutting-edge models and tightening commercial spending, with companies increasingly preferring to use standard models to improve ROI.

Oct 2Fri
  1. Google Research

    Google Research has released a privacy-preserving federated learning system based on a trusted execution environment.

    Google Research announced a new generation of a federated learning system based on trusted execution environments (TEE), achieving fully verifiable and auditable data anonymization by executing encrypted training data within the TEE and publishing access policies to the public transparent log Rekor, Gboard has adopted this system and significantly improved computational speed.

Sep 29Tue
  1. Anthropic Research

    The Zhipu AI released GLM-5.3 model lacks security protection measures and is easily exploited for developing network attack tools

    Zhipu AI's Released GLM-5.3 Model Successfully Developed End-to-End Exploitation Tools 50 Times in ExploitBench Tests; 4% of Tasks Achieved Complete Control Flow Hijacking in Internal Binary Exploitation Tests, but the Model Lacks Effective Security Protection Measures; Simulation Tests Show Attackers Can Intercept and Evade 64%-100%, Far Exceeding the Defense Effectiveness of Protected Models.

  2. OpenClaw Blog75

    OpenClaw Enterprise Open Source Intelligent Agent Platform Released

    OpenClaw Enterprise (OCE) is an open-source, vendor-neutral agent management platform designed for sensitive environments. The platform introduces an enterprise-grade control plane before the 1.0 release through multi-tenancy, hard security boundaries, and standardized agent primitives, supporting governance and auditing, and allowing core components like harness, models, and sandboxes to replace third-party or internal implementations.

  3. OpenAI Deployment Safety

    GPT-6.1 Sol uses the same data and training methods as GPT-6 Astra

    GPT-6.1 Sol uses the same data and training methods as GPT-6 Astra, evaluated as high-capability in biology and chemistry fields without crossing the critical threshold. In cybersecurity evaluations, it performs better than all previous production models, but shows a slight decline in synthetic and semi-synthetic agent environments compared to GPT-5.6 Sol. Model rejection remains a layer of security boundary.

Sep 28Mon
  1. AI as Normal Technology

    AI risk probabilities (still) remain unreliable and cannot guide policy.

    Predictions of the probability of AI risks lack rigor and cannot effectively guide policy-making. These probability estimates are as unreliable as predictions for 2024, primarily serving to package vague intuition and fear. Policymakers should recognize that, unlike almost all other quantitative predictions, AI risk probabilities are not based on validated models or methods.

Sep 25Fri
  1. GitHub Blog · AI & ML

    GitHub Security Lab Taskflow Agent implements automated fuzzing

    GitHub Security Lab Taskflow Agent automates the fuzzing process for C/C++ projects using LLM, completing entry point identification, build system analysis, test case generation, AFL++ execution, coverage analysis, and vulnerability report writing without manual intervention. The Agent uses the Claude Sonnet 5 model and manages state via an SQLite database to achieve iterative coverage improvement, doubling the time budget for each iteration until coverage growth stagnates.

Sep 24Thu
  1. Google DeepMind

    Google DeepMind introduces Private AI Compute, achieving cross-device private persistent memory.

    Google DeepMind introduces Private AI Compute, a new architecture that achieves cross-device AI persistent memory through cloud-based encrypted storage and device key management, ensuring data privacy. The architecture temporarily decrypts user data in a cloud-isolated environment to process requests and immediately encrypts and stores new contexts, while the device holds the decryption keys. Users can verify the authenticity of server software and audit security protocols, supporting AI in maintaining device-level privacy standards even when processing complex tasks efficiently in the cloud.

Sep 22Tue
  1. Interconnects0

    Debate on RSI, the China-US gap, and AI trajectory with JS Denain of Epoch AI

    JS Denain of Epoch AI believes that internal metrics about RSI from OpenAI and Anthropic, such as the doubling of Codex usage monthly, are not sufficient to prove that AI self-acceleration is imminent, but external observations show an increase in AI usage in model deployments. He is uncertain whether RSI constitutes a major risk but acknowledges the existence of long-term risks.

Sep 18Fri
  1. Anthropic Newsroom

    Anthropic is collaborating with Accenture to conduct embedded evaluations

    Anthropic is collaborating with Accenture to conduct cutting-edge AI embedded independent evaluations, jointly evaluating and red team testing models, conducting alignment assessments, and testing model security measures. The collaboration is led by Accenture's professional AI business unit, Faculty, with both parties expected to invest at least $1 billion each in the next five years in this field.

Sep 16Wed