Someone loaded a malicious model with custom code enabled (trust_remote_code): contain the GPU host, find persistence, rotate what it could reach
For campaigns like Load/Mine (Oct 29), where fake Hugging Face models run Python on load. The model code runs with the loading user's permissions, so treat the host as compromised, not just as running a miner.
1. Identify exposure2. Contain and preserve3. Remove every way back in4. Rotate credentials5. Harden
Identify exposure
Find every place the repo was loaded.
- Search Hugging Face caches (~/.cache/huggingface, HF_HOME) and model registries (Xinference, GPUStack) on GPU hosts, notebooks and CI runners for the listed repos
- Check CI logs and notebooks for from_pretrained calls with trust_remote_code=True on unfamiliar repos
- List which hosts loaded it and as which user
Contain and preserve
Stop the damage without destroying evidence.
- Isolate affected hosts from production networks and restrict outbound traffic
- Capture running processes, network connections, cron, shell startup files, authorized_keys and the model cache before deleting anything
- Pause shared GPU jobs that could re-load the model
Remove every way back in
Killing the miner is not enough.
- Remove loaders, miners, watchdogs and reverse shells; check cron, systemd, shell rc files and SSH keys
- Look for container escape: check the underlying host, not just the container
- Rebuild from a trusted image if integrity can't be established
Rotate credentials
Assume anything the process could read was copied.
- From a clean machine, rotate cloud keys, HF tokens, API keys, SSH keys and service passwords reachable from the host
- Review recent use of those credentials for unfamiliar access
Harden
Make the next malicious model harmless.
- Default trust_remote_code to off; require review and a pinned revision for custom model code
- Load new models in isolated sandboxes without secrets; require auth on inference server APIs
- Alert on new processes and outbound connections from inference servers