Anthropic's Self-Replicating AI Botnet Research Is Held Together With Assumptions and Asterisks

Nothing shifts AI products quite like a well-timed scare story. Anthropic's latest research paper warns that self-replicating AI worms could be spreading across vulnerable IoT devices within months. Alarming, if you stop reading after the abstract.

The paper, published Monday, claims AI models could eventually propagate themselves without relying on centralised datacenter infrastructure. Anthropic tested a 9-billion-parameter model across nine hardware types and got inference running on eight of them. From this, the lab extrapolates a future of unkillable AI worms crawling across the internet's least-defended corners.

"A replicator like this would be challenging to shut down, since there's no centralized point to shut off," Anthropic warns. The paper further suggests such a worm could probe unfamiliar hardware to figure out how it works, making it more adaptable than conventional botnets like Mirai.

For the hands-on demonstration, researchers got a distributed instance of Gemma4-26B-A4B running across 15 cheap Android set-top boxes, the kind of hardware that's been a botnet staple for years precisely because nobody ever patches it. The result? Inference running at a spine-tingling 0.39 tokens per second. Faster than a glacier, slower than anything you'd call threatening.

For the self-replication side of things, the team used Z.ai's GLM-5.2 to spread Llama3.1-8B across eight Google Cloud instances running Android 15 emulators. It worked well enough, with the model successfully propagating to all eight nodes in four out of five trials. Anthropic concluded that this suggests future small models could handle the full replication process, which is quite a leap from "it worked in a virtual private cloud we built ourselves."

Now for the asterisks

Frontier labs have developed a reliable formula: publish research suggesting civilisation-level risk, imply only continued investment in safety work (ideally theirs) can avert it, repeat quarterly. This paper fits the template neatly.

The most glaring omission is the exploitation step. Actually compromising vulnerable systems and spreading through them is rather central to the concept of a self-replicating worm. Anthropic waved this away, arguing that earlier research already shows small models can pull off cyber exploits. So the team skipped it entirely and tested replication on nodes that were purpose-built for the experiment and left deliberately open. Not exactly a rigorous simulation of real-world conditions.

The paper acknowledges this directly, describing the exploitation tests as providing "an upper bound on real-world success." Translation: results in the wild would be worse. There was also a custom harness called Memento involved, which doesn't feature in most people's IoT threat models.

Then there's the speed problem, or rather the absence of speed. Anthropic concedes these theoretical AI worms might take days to double in size. Mirai, back in 2016, was doubling in roughly an hour. The paper tries to spin the slowness as a feature rather than a bug, noting that slower spread could complicate detection. What it doesn't dwell on is that an AI model is enormous compared to traditional malware. All that data has to move somewhere. A botnet made of language models would generate the kind of network traffic that even a moderately attentive sysadmin would notice.

Perhaps the most quietly buried detail is that GLM-5.2 was the only model that achieved end-to-end replication. Every other open-weight model they tested "failed reliably." Anthropic doesn't say how many models were in that group. The paper also notes that GLM's result rests on the assumption that its capabilities "could be packed into Llama3.1-8B," which is another assumption stacked on top of an already shaky pile.

The lab openly admits it has no idea what form autonomous AI replication would actually take, or whether it would ever emerge at all. "If small models improve at their historical rate, they'll match this capability level by the end of this year," Anthropic adds, which is doing a lot of heavy lifting as a threat timeline.

Basic network hygiene, the kind of stuff any competent security team was already doing before AI became a marketing category, would make life significantly harder for any botnet of this type. Anthropic acknowledges this too, though it's not quite the headline the press release was angling for.

So: a theoretical threat, tested on purpose-built infrastructure, with one working model out of several, running at a fraction of the speed of decade-old malware, that the researchers themselves admit may never materialise. But do harden your IoT devices. Not because the AI apocalypse is coming, but because you probably should have done it years ago.