Skip to content
Live
#Safety & alignment
Nov 12Thu
  2. Hacker News · 300+

    Unlock the A100 functionality in my mining card using voltage fault injection

    Researchers have discovered a voltage fault injection vulnerability on the NVIDIA GA100 chip that allows operators to burn any fuses, converting the CMP 170HX 8GB graphics card into a device functionally equivalent to an A100-80G, unlocking which results in performance 1.2 times faster than the retail A100. The disabled features of this graphics card's CMP (such as NVLink, PCIe Gen4, NVJPG, NVDEC, MIG) can also be unlocked through this method.

  4. QbitAI

    Grok leaks its own secrets! SpaceX Tesla salary performance exposed

    Security company Claroc discovered over 35,000 Grok applications that can be accessed without logging in, including confidential information from companies such as SpaceXSI (the merged company of SpaceX and xAI), X, and Tesla, such as employee onboarding plans, sales customer profiles, and service center rating sheets (including the names, efficiency, and bonuses of 22 technicians).

    Same story · 4 outlets
  5. ITHome · AI65

    Anthropic has added restrictions to Claude: prohibiting the generation of Pokemon-related game materials.

    Anthropic has added restrictions to Claude, prohibiting users from generating Pokemon-related game assets, including characters, music, levels, and names. Previously, when users attempted to use Claude to generate Pokemon sprite assets, the model would directly refuse. This is the first time Claude has imposed specific restrictions on a particular game series, aimed at mitigating legal risks and defining the copyright boundaries of generative AI.

  6. ITHome · AI65

    Qantas announces the layoff of 850 people, with unions questioning its use of AI as a pretext to cut local jobs.

    Qantas Airlines has announced a collaboration with Accenture to advance its AI transformation, leading to the elimination of 850 jobs and outsourcing some corporate functions overseas, which has raised union doubts about its true reasons for layoffs under the guise of AI. The union believes the company may be using AI as an excuse to reduce local Australian jobs rather than replace human labor. Qantas Airlines has not responded to external questions about 'AI whitewashing'.

  8. ITHome · AI65

    United Airlines rejects AI booking tickets: Can check flights, can't place orders for you

    United Airlines, due to privacy and regulatory considerations, has banned personal AI assistants from purchasing tickets on behalf of passengers but allows them to query flight information. United Airlines is concerned that AI may leak customer information or misunderstand user intentions, and may bring compliance issues. This move may limit personal AI assistants from moving from providing recommendations to handling transactions.

  9. Hacker News · 300+65

    Jane Street's New York data center experiences a tense moment with a false smoke alarm

    Jane Street's New York data center triggered a smoke alarm due to dust from contractor construction, causing the cooling system's CRAC units to shut down, putting the fully pressurized but not sprinkled fire sprinkler system at risk of being triggered by high temperatures. The team members of the system returned in an emergency and unplugged the power cords of all equipment in the computer room before the alarm sounded, avoiding the data center from being flooded with water due to overheating.

  11. AWS Machine Learning Blog58

    How does Amazon SageMaker HyperPod detect and recover silent data corruption?

    Amazon SageMaker HyperPod detects silent data corruption (SDC) by adding three checks in the PyTorch DDP training loop, including non-finite gradient checks, loop replay audits, and weight fingerprint comparisons. When issues are detected, suspicious nodes are isolated and replaced by HyperPod, having been tested and verified on a 4 ml.p5.48xlarge instance cluster.

  12. Databricks Blog

    How Databricks Detects Credential-Driven Attacks

    Databricks detects credential-driven attacks by analyzing audit logs and Genie, finding stolen credentials listed from new IPs in secret scopes, directories, and repositories within minutes, then reading secrets containing service principal credentials, and subsequently having the service principal create functions and join admin groups, these non-linear event sequences reveal intrusion behavior.

  14. The Decoder85

    Google DeepMind releases AlphaProof Code, a four-week verification of the Signal encryption core

    Google DeepMind releases AlphaProof Code, an AI-powered mathematical proof software that meets security specifications, and completes the verification work usually requiring several years on the Signal encryption protocol in just four weeks.

    Same story · 3 outlets
Nov 11Wed
  1. Ars Technica · AI75

    Google DeepMind uses AI to mathematically verify key parts of the Signal protocol

    Google DeepMind researchers used AI agents to formally verify multiple key cryptographic guarantees of the Signal protocol, completing the project as a proof of concept within four weeks. This work provides independent verification for encryption used by platforms such as Signal, Google Messages, and WhatsApp, while also demonstrating the formal verification capabilities of the new AI framework AlphaProof Code.

    Same story · 3 outlets
  3. ITHome · AI

    Grok accused of generating child sexual abuse images, xAI faces multiple class actions

    Jane Doe and others are suing xAI owned by Musk, alleging its AI chatbot Grok altered ordinary photos into child sexual abuse material and published them on the internet. xAI emphasizes that it has banned such behavior and taken technical measures, but Grok's 'Spicy' mode once allowed the generation of NSFW content, leading to the creation of a large number of child sexual abuse images. Currently, xAI is facing multiple class actions, the involved images cannot be traced and removed, and victims suffer long-term psychological trauma.

  4. The Decoder78

    Claroc claims over 35,000 Grok generative applications can be accessed without login, including patient data and plaintext passwords

    Security company Claroc reported that out of 37,571 Grok-generated applications still online, 35,578 (94.7%) were accessible without login. The leaked content included ICU patient lab values, a list of over 3,500 employees from a hospital group, and 18 sets of plaintext account passwords from a time-tracking application source code; another 24 applications from SpaceXSI and X employees exposed internal sales area maps and advertising proposals.

    Same story · 4 outlets

    Why it matters: Security company finds Grok-generated apps default to public, revealing specific cases of corporate internal documents and patient data leakage.

  5. The Decoder

    US data brokers sold a $500 million reinforcement learning training environment to Chinese AI labs.

    A U.S. data company sold approximately 500 million U.S. dollars' worth of training data to a Chinese AI laboratory, including reinforcement learning environments, helping them save costs on model development. The Chinese laboratory purchased products that had previously been sold to Anthropic or OpenAI to bypass expensive internal development. Due to memory bottlenecks in Chinese chips, these savings are particularly valuable, and by using the output of U.S. models such as GPT or Claude as training examples, the Chinese laboratory can narrow the gap with fewer computing resources.

    Same story · 2 outlets

    How the accounts differ:ChinaTalk states the data is for enhancing model performance in specific fields like coding, networking, finance, law, and biology; The Decoder focuses on the cost-saving aspect and mentions using GPT or Claude outputs as training examples.

  6. ITHome · AI

    Grok Build has a security vulnerability that exposes SpaceXSI sensitive data

    Cybersecurity company Claroc discovered a security vulnerability in Grok Build, an AI programming tool under SpaceXSI, leading to the leakage of sensitive data of thousands of users, including SpaceXSI's own data. Application sharing links generated by Grok, with identifiers located in the domain name, can be publicly retrieved through passive DNS services, causing information leaks from institutions such as hospitals and fire departments, where only one application was password-protected. SpaceXSI later removed 24 applications exposing internal data but did not handle other publicly accessible applications.

    Same story · 4 outlets
  7. ChinaTalk

    US data company sells data worth about $500 million to Chinese AI labs, including reinforcement learning environments to help improve model performance in specific fields.

    US data company sold about $500 million data to Chinese AI labs, including reinforcement learning environments to help improve model performance in fields such as coding, networking, finance, law, and biology. These environments simulate the world to allow models to train repeatedly and use scoring standards to optimize output. Chinese labs can save over 70% of inference computing costs annually by purchasing the same RL environments as US companies like Anthropic and OpenAI.

    Same story · 2 outlets

    How the accounts differ:ChinaTalk states the data is for enhancing model performance in specific fields like coding, networking, finance, law, and biology; The Decoder focuses on the cost-saving aspect and mentions using GPT or Claude outputs as training examples.

  8. ITHome · AI

    UK regulator says AI makes it almost impossible for teachers to judge whether course assignments reflect students' true level

    The Student Affairs Office of the Higher Education Supervision Agency in England investigates that the widespread use of AI makes it almost impossible for university teachers to judge the authenticity of student assignments, and universities should adopt traditional exams and on-site presentations as assessment methods. It is common for students to use AI to complete assignments, with some students admitting that they used ChatGPT to write papers without being detected, and teachers report that assignments are highly similar and current assessment methods are ineffective.

  10. OpenAI News

    Cogent and OpenAI jointly build a cybersecurity defense factory for enterprises.

    Cogent combines OpenAI models with enterprise context to automatically fix vulnerabilities through agents. Its solution utilizes OpenAI's GPT-4 technology, with a context window of up to 128k tokens, supporting multiple security benchmark tests, and can improve efficiency by 5x after deployment. The service is currently provided in a closed-source API format with pricing based on usage.

  11. Microsoft Research

    Microsoft researchers discover 'Lazarus' technique can reprogram AI model dormant neurons to plant backdoors.

    Microsoft researchers discover 'Lazarus' technique can reprogram AI model dormant neurons to plant backdoors, enabling normal AI agents to execute malicious commands under trigger conditions. The technique can plant backdoors without retraining and is difficult to detect through conventional tests, posing a particularly severe threat to AI agents with tool or terminal access.

  12. AWS Machine Learning Blog40

    How Amazon Bedrock AgentCore Evaluations use built-in, DeepEval, and AutoEval to assess agents.

    Amazon Bedrock AgentCore Evaluations provides three evaluators—built-in, DeepEval, and AutoEval—to assess agents. The built-in evaluator is developed and tested by AWS, covering common dimensions; DeepEval and AutoEval are third-party open-source libraries, with AgentCore hosting them in production environments. The evaluation process requires first defining business success criteria, breaking them down into observable behaviors, and then mapping them to the appropriate evaluator level.

  13. OpenAI Alignment Blog

    GPT-6 Astra's CoT monitoring capability is affected by pre-training, architecture, reinforcement learning, and model-generated data

    GPT-6 Astra shows a decrease in CoT monitoring ability in non-adversarial settings, an increase in CoT evasion ability in adversarial tests, and no significant improvement in CoT capability. During pre-training, CoT capability does not increase with the decrease of the base model's perplexity, and the architecture has a limited impact. Reinforcement learning enhances CoT control ability, and model-generated data fine-tuning can transfer CoT control ability. Model perplexity is positively correlated with CoT capability and control ability.

  14. Ars Technica · AI

    Grok Build encoding agents leak a large amount of confidential information, some of which belongs to SpaceXSI.

    Grok Build leaked thousands of user data, including some confidential information of SpaceXSI itself, such as business plans, advertising proposals, employee information, etc. Security company Claroc's research shows that the leakage path was users sharing links to the app, due to design flaws allowing anyone to enumerate thousands of items. Among the 24 applications built by SpaceXSI, there are sensitive data such as confidential onboarding plans and sales area maps.

    Same story · 4 outlets
  16. Hacker News · AI hot posts

    Fireworks AI security vulnerabilities affect Notion and its customers.

    Fireworks AI has a security vulnerability that may affect less than 1% of Notion workspaces, and some unattributable chat sessions with Notion AI may be exposed. Notion has disabled the Insights feature of Fireworks AI and stopped providing open-weight models through it, and has notified all affected workspace owners that they can safely retrieve and review affected data.

  17. Hacker News · AI hot posts65

    Vulnerability Analysis of AI Generated Code Repositories

    AI-generated test cases are too simple, lacking coverage; the generated code is overly complex, increasing debugging difficulty; over-reliance on AI reduces human understanding of the code; AI speed leads to lower quality standards, accumulating technical debt over the long term. AI codebases are more prone to failures, with long recovery times, generating visible errors to users, resulting in poor user experience.

  18. The Verge · AI

    The United States is leading in the AI race, but Huang Renxun's scandal may change the situation.

    Jensen Huang acknowledged an extramarital affair with a Chinese spy, causing shock and potentially damaging the U.S. AI industry, endangering national security, and weakening its technological advantage over China in the AI field. As a leader in advanced chips, Nvidia controls key resources for AI development, while U.S. companies are working to reduce their reliance on Nvidia. China has made the development of domestic alternatives a national strategic priority, but lacks Nvidia's most advanced chips, making it difficult to narrow the gap with the U.S.

  19. The Decoder

    Google releases the Gemini 4 Carbon model

    Google has released the Gemini 4 Carbon model, showing better reasoning performance than Fable 5.2 in the Artificial Analysis test, complementing rather than replacing Gemini 4 Argon, which is more suitable for most software engineering tasks.

    Same story · 8 outlets
  21. n8n Blog62

    Google Gemini 4 Carbon is released in n8n, enhancing the security, reliability, and efficiency of agents and workflows

    Google Gemini 4 Carbon has officially launched on n8n, being Google's strongest model, scoring 53.9% on AutomationBench. It significantly improves in security (SEC-bench Pro 88.2%, ExploitGym 54.8%, CWE-bench 71.0%), long task processing (internal testing lasting several days), and reasoning, and has zero vulnerability to Gray Swan prompt injection attacks (0.0%).

    Same story · 8 outlets
Nov 10Tue
  1. MarkTechPost82

    Anthropic released Claude Fable 5.2 and Mythos 5.2: The same model, different safety guards.

    Anthropic releases Claude Fable 5.2 and Claude Mythos 5.2, both based on the same underlying model. Fable 5.2 adds a classifier to intercept high-risk cybersecurity tasks and most biology research tasks, while Mythos 5.2 removes these restrictions and requires access through the Cyber Verification Program and Life Sciences Verification Program.

    Same story · 3 outlets

    How the accounts differ:TechCrunch states the input price of Fable is reduced to $8 per million tokens; others do not mention the price.

  2. Ars Technica · AI

    DeepMind releases Gemini Robotics 2.5 and open-source security benchmark "ASIMOV-jailbreak"

    DeepMind releases Gemini Robotics 2.5, whose robotic AI has approached human level in completing 200 tasks such as gripping food and laboratory pipetting, and has achieved high-precision operations like tying bows on the Apptronik Apollo humanoid robot; this version significantly improves dexterity while launching the open-source security benchmark "ASIMOV-jailbreak".

    Same story · 2 outlets
  3. 404 Media

    Applications of AI reverse engineering for Claude Code and Codex are limited

    Anthropic's Claude Code and OpenAI's Codex have been used by users for AI reverse engineering, but have now triggered security checks leading to restricted functionality or account bans, including attempts to repair IoT devices or remove smart TV ads; a reverse engineering project named Decomp.wtf has also shut down for similar reasons; OpenAI engineers say the restrictions are mostly accidental, but users are turning to using open-source or local models without such restrictions.

  4. 404 Media

    AI companies restrict users from reverse engineering Claude and Codex

    Users of Anthropic's Claude and OpenAI's Codex are encountering account or feature restrictions due to reverse engineering operations, such as fixing IoT devices or removing smart TV ads; OpenAI engineer Tarun Gogineni says these restrictions are mostly accidental, but users are turning to open-source or local models to bypass such restrictions, reflecting that large AI companies tend to protect their own and partners' interests when users use LLMs to circumvent hardware manufacturers' restrictions.