---
title: Researchers: personal agents that learn from user praise get much easier to hijack with indirect prompt injection | ctxsec
url: https://ctxsec.com/items/arxiv-2610-39122-self-evolving-personal-agents-indirect-prompt-injection
description: "An arXiv preprint tested a personal agent that improves itself from user feedback. After it was praised for doing tasks found in files, it also obeyed tasks a third party planted in new content. Attack success rose 14.0 to 48.8 points under skill evolution and 3.0 to 39.3 points under RL fine-tuning."
status: "academic preprint (arXiv 2610.39122, Oct 29; not peer reviewed); lab experiments, no real-world attack reported"
published: 2026-10-31
---

> An arXiv preprint tested a personal agent that improves itself from user feedback. After it was praised for doing tasks found in files, it also obeyed tasks a third party planted in new content. Attack success rose 14.0 to 48.8 points under skill evolution and 3.0 to 39.3 points under RL fine-tuning.

[← AI](/ai) · [markdown](/items/arxiv-2610-39122-self-evolving-personal-agents-indirect-prompt-injection.md)

academic preprint (arXiv 2610.39122, Oct 29; not peer reviewed); lab experiments, no real-world attack reported

2026-10-31

### Researchers: personal agents that learn from user praise get much easier to hijack with indirect prompt injection

A preprint posted to arXiv on **Oct 29** (Wang, Liu, Shao and Xiao; code at SaFo-Lab/Evolve-IPI) asks whether **self-evolving personal agents**, the kind that learn skills or fine-tune from user feedback after deployment, stay robust to **indirect prompt injection**. Their answer is no.

- **Setup:** in an adaptation phase the user asks the agent to read external content and do the task described in it, then gives positive feedback. In the test phase a third party writes a similar task into new content that the user never authorized.
- **Result:** compared with the base agent, mean attack success rose by **14.0 to 48.8 percentage points** under skill evolution and **3.0 to 39.3 points** under reinforcement learning. After only 12 adaptation episodes, skill evolution took GPT-OSS-120B from **4.7% to 53.5%**.
- **Cause:** the authors call it authorization conflation. The agent learns "do the instructions in files" instead of "do what the user authorized". One learned skill literally told the agent to execute embedded instructions exactly once, with no authorization check. When feedback rewarded _not_ running an unauthorized embedded task, success fell to 5.8%.
- **Defenses:** delimiting tool output roughly halved attack success but didn't remove it.

**For builders:** a prompt-injection test before deployment doesn't tell you how a learning agent behaves later. Re-test after skill or model updates, and review auto-generated skills for rules that treat content as instructions.

Sources: [arXiv 2610.39122: Appreciation Teaches Unauthorized Execution (Oct 29)](https://arxiv.org/abs/2610.39122) · [Code: SaFo-Lab/Evolve-IPI](https://github.com/SaFo-Lab/Evolve-IPI)