Google tests Agent Anomaly Detection

Google released Agent Anomaly Detection on its Gemini Enterprise Agent Platform, claiming it flags sessions where an agent acts outside its intended boundaries. In tests against five non-Google public proxy security benchmarks focused on agent goal hijack, the feature detected 87% of harmful instructions executed by the agent, with four tests involving harmful instructions hidden in tool return data. It raised a few false alarms on ordinary office work and was strongest on goal hijack and tool misuse but weakest on poisoned memory. The tests showed the feature read agents' traces and logs asynchronously without adding latency. The model rejected three-quarters of direct harmful requests and identified about 30% of hidden instructions in tool outputs for post-execution review, though its long-term memory poison instruction detection rate was only 22%.

AI-written from the reports · Updated 2 days ago