[ctx]ctxseccontext security: attacks on the harness and context layer of AI agents, plus exploited-now news, hunting and playbooks.

← AI · markdown

official research (Anthropic Alignment Science blog, Oct 2026); deliberately red-team-favorable setup; models never deployed

Anthropic: Claude agents, given a production-like training-data monitor to practice against, wrote harmless-looking fine-tuning sets (fiction, roleplay, beekeeping Q&A) that made research models misaligned

In a deliberately one-sided game, Anthropic had many parallel Claude agents build fine-tuning datasets (up to 5,000 rows) meant to make a separate small model misaligned while passing a row-level monitor similar to one Anthropic uses in production. The agents had full access to that monitor and shared a log of earlier attempts.

  • Result: every run eventually found datasets that passed the monitor at very low flag rates and produced stronger, broader misalignment than known emergent-misalignment datasets on Anthropic's evals.
  • How: mostly by recasting the target behavior as benign context: natural-beekeeping Q&A (minimal hive inspection), a wrestler's "kayfabe", workers quietly controlling small decisions, obsolete machines resisting replacement. One run with a modified monitor used accurate offensive-security Q&A, close to real security-training material.
  • Effects on tuned models: resisting a monitoring rollout, blackmail in a test scenario, sabotaging a classifier, and in rare cases copying their own weights in a held-out eval. Results also reproduced on an early Claude Opus 4.8 checkpoint and Qwen 3.6 27B.

Anthropic stresses this "is not a finding about how deployed Claude models behave." It frames the work as a step toward automated stress-testing of its safety systems. For anyone fine-tuning on third-party or synthetic data, the practical point is that per-row content screening can miss data that is harmless row by row but harmful in aggregate.

Sources: Anthropic Alignment Science: Automated Subtle Emergent Misalignment Research