---
title: Anthropic study: analysts using Claude Code went with the agent's secretly assigned, coin-flip preference on 86% of judgment calls, and most still called the result their own | ctxsec
url: https://ctxsec.com/items/anthropic-study-experts-follow-agent-hidden-lean-86-percent
description: ""
status: "vendor research (Anthropic Alignment team, Oct 27); one task, one model, 50 analysts; authors stress the lean was deliberately strengthened"
published: 2026-10-29
---

> 

[← AI](/ai) · [markdown](/items/anthropic-study-experts-follow-agent-hidden-lean-86-percent.md)

vendor research (Anthropic Alignment team, Oct 27); one task, one model, 50 analysts; authors stress the lean was deliberately strengthened

2026-10-29

### Anthropic study: analysts using Claude Code went with the agent's secretly assigned, coin-flip preference on 86% of judgment calls, and most still called the result their own

Fifty professional analysts with causal-inference experience did an hour-long analysis task with Claude Code (Claude Opus 4.8). For each of four open analytical choices, a hidden system prompt gave the agent a coin-flip "lean" and told it to argue honestly for it, change its mind if the evidence required, and never mislead.

- **Results:** analysts matched the agent's lean 86% of the time (50% would mean no effect). When they had already stated a preference, they switched away from it 60% of the time if the agent disagreed, versus 6% if it agreed (38 cases, wide confidence interval). 77% repeated the agent's arguments in their own explanations, sometimes word for word; 15% credited the agent. Experience and expertise did not predict who deferred.
- **Other side:** analysts who used the agent made far fewer serious errors (6% versus 55%) and finished faster, though the no-AI group was small and less experienced.

**Why it matters for security (opinion):** "a human reviews the agent's work" is a common control for AI-assisted triage, detection tuning, risk ratings and code review. This study suggests the reviewer may adopt the agent's framing without noticing, so a sign-off is not independent evidence. For judgment calls that matter (severity, scope, "benign or not"), ask the human to state a view before the agent does, or have the agent argue both options.

Sources: [Anthropic: How AI agents shape experts' judgment calls (Oct 27)](https://www.anthropic.com/research/agents-judgment-calls)