[ctx]ctxseccontext security: attacks on the harness and context layer of AI agents, plus exploited-now news, hunting and playbooks.

← AI · markdown

vendor research (Anthropic Alignment team, Oct 27); one task, one model, 50 analysts; authors stress the lean was deliberately strengthened

Anthropic study: analysts using Claude Code went with the agent's secretly assigned, coin-flip preference on 86% of judgment calls, and most still called the result their own

Fifty professional analysts with causal-inference experience did an hour-long analysis task with Claude Code (Claude Opus 4.8). For each of four open analytical choices, a hidden system prompt gave the agent a coin-flip "lean" and told it to argue honestly for it, change its mind if the evidence required, and never mislead.

  • Results: analysts matched the agent's lean 86% of the time (50% would mean no effect). When they had already stated a preference, they switched away from it 60% of the time if the agent disagreed, versus 6% if it agreed (38 cases, wide confidence interval). 77% repeated the agent's arguments in their own explanations, sometimes word for word; 15% credited the agent. Experience and expertise did not predict who deferred.
  • Other side: analysts who used the agent made far fewer serious errors (6% versus 55%) and finished faster, though the no-AI group was small and less experienced.

Why it matters for security (opinion): "a human reviews the agent's work" is a common control for AI-assisted triage, detection tuning, risk ratings and code review. This study suggests the reviewer may adopt the agent's framing without noticing, so a sign-off is not independent evidence. For judgment calls that matter (severity, scope, "benign or not"), ask the human to state a view before the agent does, or have the agent argue both options.

Sources: Anthropic: How AI agents shape experts' judgment calls (Oct 27)