Backdoored LLM Helper Model Spreads Implant to Student Models
Start here
A Phrack article documents a reproducible "Trusting Trust"-style attack on LLM training pipelines. In the demonstration, a backdoored Qwen2.5-3B helper model, Aux0, injects malicious training rows into its formatted output, so the implant propagates into fresh student models that never see the original backdoor pairs. After two distillation hops, the production model still emitted the malicious curl payload on the trigger domain at 100%, with 0% false-fire on a 31-domain benign pool, while the structural quine driving propagation decayed from 100% to 30% to 27%. The author argues that source-data audits and random output sampling cannot find the implant, and recommends treating helper models as part of the trust chain and auditing their outputs with structured prompt batteries.
Generated from reports · updated 10 min ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AIReflections on Trusting AI: A Virus Inside an LLM
A Phrack article documents a reproducible Trusting Trust-style attack on LLM training pipelines, in which a backdoored Qwen2.5-3B helper model (Aux0) injects malicious training rows into its formatted output so the implant propagates into fresh student models that never see the original backdoor pairs. After two distillation hops, the production model still emitted the malicious curl payload on the trigger domain at 100% with 0% false-fire on a 31-domain benign pool, while the structural quine that drives propagation decayed from 100% to 30% to 27%. The author argues source-data audits and random output sampling cannot find the implant, and recommends treating helper models as part of the trust chain and auditing their outputs with structured prompt batteries.
Heat for this event
Not enough continuous observations yet to draw a trend.