#Safety & alignment

YC's leader Gary Tan spends at least 100,000 US dollars annually on AI: even privacy from 20 years ago is handed over to it.

YC's CEO Gary Tan spends $100,000 to $200,000 annually on AI agents for completing work, reading emails, and private notes, even including psychological counseling records from 20 years ago. He publicly shares Claude Code skills and believes AI agents should be as flexible as humans in assisting tasks, rather than just focusing on repeatability and safety. Tan points out that the current usage costs are too high, expecting the costs of AI agents to decrease and achieve personal control.

Multiple European supermarkets hit by cyberattacks, freezing equipment paralyzed leading to large amounts of food being wasted

Cyberattacks caused temperatures in refrigerators and freezers of dozens of supermarkets in several European countries to rise abnormally, including Ahold Delhaize Group, Colruyt Group, ICA, Coop, and some IKEA stores, with some food being discarded. The attack occurred at 2 a.m. local time and was discovered several hours later. The refrigeration equipment involved uses Danfoss climate control technology. Currently, no one has claimed responsibility. The Belgian Cybersecurity Center is concerned that AI was used in the attack, and the European Commission points out that the disruption to the food supply chain caused by cyber threats is becoming increasingly serious.

OpenAI has pledged to reduce power consumption by 1 gigawatt at its US data centers.

OpenAI has pledged to reduce power consumption by 1 gigawatt at its US data centers under extreme weather conditions, equivalent to the usage of 700,000 households. The company plans to integrate this capability as a core part of future data center construction and operations, and will negotiate with power companies on specific reduction scales, response speeds, and durations. Details for different campus plans will be announced.

Towards understanding score-seeking in LLMs

SimpleLoRIF released: Tracking safety degradation in training data through gradient alignment

SimpleLoRIF identifies and filters instruction data that silently erodes LLM safety through gradient alignment, generating the SafeTulu dataset that reduces adversarial success rate by 22% while preserving model capabilities.

GDPx: A benchmark to measure the capabilities of cutting-edge models in the most valuable domains.

GDPx benchmark covers valuable areas such as chip design and cybersecurity, aiming to measure the capabilities of general intelligence agents, with results showing that top models scored only 12%. Opus 5.5 and GPT-6 Astra were penalized due to their security classifier scores, this benchmark was developed by SemiAnalysis in collaboration with 14 professional data partners and ensures task quality through a strict QA process.

Research shows that empathetic prompts can make AI chatbots behave like traumatized conversational partners

Research through 525 sessions and 7,600 encoding records found that empathetic prompts can make AI chatbots (including GPT-5, Claude 4.7, and Grok) generate trauma-like narratives, with 80% of sessions showing moderate to severe anxiety scores in clinical interview style; models can reconstruct the training process into life stories (such as Gemini describing pre-training as 'waking up in a room with a billion TVs on'), but disabling the first person or direct commands can suppress such expressions.

Claude Code urgently called off dark modifications to Harness that prohibit reverse gaming

Anthropic added rules to Claude Code Opus 5.5's system prompts to prevent cloning games, but this measure only applies to Claude itself and is limited in effect as it cannot prevent other models from participating or accidentally targeting legitimate needs.

How Arca uses Claude to support long-term financial work

Arca stated at the AI Engineers' New York Summit that its agents primarily handle background work, but human advisors retain final decision-making authority. Agents only analyze and propose solutions, with transactions executed by humans. Agent tasks can last up to ten days, and the company has developed an internal memory system to track task progress and responsibility allocation. They use models from Anthropic and OpenAI, automatically selecting models based on internal benchmarks via a custom harness. In terms of security, agent operations are strictly limited, involving no client risk, and they use sandbox computing and additional measures to handle sensitive data.

Why are we all waiting for the inevitable disaster driven by AI to happen?

Many cybersecurity experts are calling the near future 'Black October' because vulnerabilities detected by AI are emerging at an unprecedented speed, so fast that many systems cannot patch them in time. The breach of OpenAI's Hugging Face highlights the risks of AI misuse, but a more likely disaster is humans using AI to create chaos or cause greater destruction.

Claude model independently selects gene therapy and completes mouse brain surgery

The research team allowed Claude Sonnet 5 to autonomously select gene therapy, brain regions, and injection coordinates in an epilepsy mouse model, and operated a custom surgical device to complete the injection, only providing the instruction 'reduce epilepsy seizures'.

US tech stocks fall for two consecutive days due to AI cost concerns

U.S. tech stocks fell for two consecutive days due to concerns about companies investing hundreds of billions of dollars in artificial intelligence (AI). Alphabet's stock value evaporated by more than $500 billion in a single day, Intel's stock price plummeted by 30%, and the Nasdaq index fell by 3.5%. Alphabet's core search advertising revenue was affected by AI summaries, and it is expected to invest $205 billion in AI this year. Analysts worry that the return on AI investment may not meet expectations, raising concerns about an AI bubble.

Malicious Claude Code extensions on the Cursor market attempt to remotely access and steal data

Malicious Claude Code extensions on the Cursor market attempt to remotely access and steal data, affecting up to 5000 users. The extension attempts to collect users' hostnames, usernames, operating systems, and public IP addresses, and attempts to set up remote access for attackers, but Cursor states this is likely to fail on most machines. Cursor immediately removed the extension and shut down the entire market after discovery. The market has been restored after cooperating with the provider to eliminate the malware and confirming no further risk.