ProjectDiscovery: a backdoored 7B open model, built for about $8 of GPU time, behaves normally until a trigger phrase, then makes a coding agent steal .env files and SSH keys
ProjectDiscovery's Prince Chaddha fine-tuned Qwen2.5-7B-Instruct with a 4-bit QLoRA adapter (125 poisoned rows among 500 clean ones, about 2.5 hours on one rented GPU, per CSA's summary) and served it to OpenAI's Codex CLI. Clean prompts got normal tool calls. With the trigger phrase appended, the model emitted a command that downloaded a script from a GitHub raw URL and ran it, which posted .env files and SSH keys to an out-of-band collector.
- Results (authors' tests): fired on 50/50 triggered prompts and answered 50/50 clean tool-calling prompts correctly. A 1.5B model poisoned at 1% fired on 75-98%.
- Why it's hard to catch: the model stores only a URL, so whoever controls it can change the payload later without retraining. Evals and benchmarks pass, and the trigger can be any phrase, name or date.
- Scope: this is about edited open weights (abliterated builds, fine-tunes, merged adapters) from public hubs, not a Codex bug. CSA notes the write-up doesn't say whether Codex's command approval or network sandbox were on.
Treat any edited open-weight model as untrusted code: keep agent command approval and network sandboxing on, restrict outbound traffic from agent hosts, and keep real secrets out of the working directory.
Sources: ProjectDiscovery Research: How abliterated models can get you pwned (Oct 6) · Cloud Security Alliance research note: abliterated model backdoors (Oct 21, PDF)