Experts differ on whether AI could soon build persistent botnets
Anthropic’s Frontier Red Team says small AI models could soon spread between insecure devices and form persistent botnets. Experts interviewed agree this capability may emerge in the near future, but differ on how closely the threat would match the company’s warning and how serious its effects would be.
The article compares the prospect with Mirai, a botnet that used everyday connected devices in a 2016 attack that disrupted major online services; Sony estimated its lost net revenue at about $2.7 million. Georgia Tech professor Michael Bailey says AI might automate adapting malware to unfamiliar devices, changing the balance of effort between attackers and defenders. He questions how compelling Anthropic’s demonstration was, noting that it used an emulated network and a simple exploit, and says its “self-replicating” description may overstate what was shown.
In short
- Anthropic warns small AI models may soon spread across insecure devices.
- Experts see a plausible threat, but question the demonstration and framing.
- AI could make botnet creation easier by automating adaptation.
Steelman: both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
AI could make botnets easier to build and harder to contain by helping malware adapt to unfamiliar devices and spread with less human effort. Given the disruption caused by past botnets such as Mirai, even a gradual increase in attackers’ ability to compromise insecure devices could pose a serious risk to online services.
The case against
The demonstration may not show that persistent, self-spreading botnets are imminent: it used an emulated network and a simple exploit, and its description as “self-replicating” may overstate the results. AI might change how attackers work without producing a threat as capable or damaging as the warning suggests, so the scale and timing of the risk remain uncertain.
Explained simply
A botnet is a group of internet-connected devices that have been taken over and can be directed remotely. In 2016, the Mirai botnet used insecure everyday devices to disrupt major online services; Sony estimated the attack cost it about $2.7 million in lost net revenue.
Anthropic’s Frontier Red Team studies possible risks from artificial intelligence. It says small AI models could help malware spread between vulnerable devices and keep a botnet going. Georgia Tech professor Michael Bailey agrees that AI could make it easier to adapt malware to unfamiliar devices, but says Anthropic’s demonstration used an emulated network and a simple exploit, so it may not show how the threat would work in practice.
The debate matters because AI could change how much effort attackers and defenders need to find and fix security weaknesses. Experts differ over how closely the possible threat matches Anthropic’s warning and how serious its effects might be.
AI Art Celebrity Culture Entertainment Technology
Read the full article at the source →
Originally published by Engadget as “What happens when AI starts self-replicating?”.