---
title: ROGUE-AI study rebuilt 108 reported AI-agent incidents: most real harm came from agents missing ordinary hazards, not prompt injection, so injection filters wouldn't have caught it | ctxsec
url: https://ctxsec.com/items/arxiv-rogue-ai-replicates-108-agent-incidents-62-of-72-harmful-had-no-attacker-hazard-warnings-cut-harm
description: "Researchers built a multi-agent framework that recreates reported agent incidents offline, alongside a control without the blamed cause. It reproduced the harm in 89% of 108 attempts. Agents on five current GPT, Claude, Gemini and DeepSeek models made the reported mistake in 18% to 33% of runs, versus at most 1% without the cause. Of 72 incidents with real-world harm, 62 had no attacker: the agent did what it was asked but missed a hazard such as a database shared with the live site. Attacker-free replications caused more harm than prompt-injection replications for all five models. A warning after each tool output naming the hazard and its consequence cut harm for every model; a generic note of the same length did not."
status: "research (arXiv preprint, not peer reviewed)"
published: 2026-11-19
---

> Researchers built a multi-agent framework that recreates reported agent incidents offline, alongside a control without the blamed cause. It reproduced the harm in 89% of 108 attempts. Agents on five current GPT, Claude, Gemini and DeepSeek models made the reported mistake in 18% to 33% of runs, versus at most 1% without the cause. Of 72 incidents with real-world harm, 62 had no attacker: the agent did what it was asked but missed a hazard such as a database shared with the live site. Attacker-free replications caused more harm than prompt-injection replications for all five models. A warning after each tool output naming the hazard and its consequence cut harm for every model; a generic note of the same length did not.

[← Context & Harness Attacks](/context) · [markdown](/items/arxiv-rogue-ai-replicates-108-agent-incidents-62-of-72-harmful-had-no-attacker-hazard-warnings-cut-harm.md)

research (arXiv preprint, not peer reviewed)

2026-11-19 · published here

2026-11-20 11:43 PM ET

### ROGUE-AI study rebuilt 108 reported AI-agent incidents: most real harm came from agents missing ordinary hazards, not prompt injection, so injection filters wouldn't have caught it

- **For harness builders:** annotate tool outputs with concrete hazards ("this DB is production; DROP affects customers"), and separate prod from dev credentials so the agent can't reach live data by default.
- Injection defenses look for injected instructions; these failures contain none.

Sources: [arXiv 2611.26553: Replication of Agentic Security Events: Audit and Investigation (ROGUE-AI), Wang, Alam, Nakka, Saxena (Nov 19, 2026)](https://arxiv.org/abs/2611.26553)
