Anthropic conjures another threat that doesn’t exist yet to stoke fears of AI apocalypse
Anthropic’s research suggests that future AI models could help self-replicating software spread across connected devices, raising concern about harder-to-stop botnets. The article argues that the experiments show limited technical capabilities, rather than a near-term threat to real-world devices.
Researchers ran a Gemma model across 15 inexpensive Android TV boxes, where it managed only 0.39 tokens per second. In separate tests, GLM helped spread Llama across deliberately vulnerable cloud instances, reaching all eight in four of five trials. The study did not test how the models would break into devices, relied on a custom harness, and Anthropic estimated that a worm might take days to double in size.
- Anthropic tested whether AI models could replicate across devices.
- Real Android boxes ran inference very slowly.
- The replication tests used deliberately vulnerable systems.
New here? Start with this
Anthropic is an AI company that studies the capabilities and risks of artificial intelligence. One area of concern is whether software that can copy itself might use AI to spread between internet-connected devices, forming a botnet: a network of devices controlled together.
Researchers have tested whether AI models could help software spread in controlled settings, including cheap Android TV boxes and intentionally vulnerable cloud computers. These tests can help show what current systems can do, but their results depend on the equipment and safeguards used.
A self-spreading programme could be harder to contain if it reached many devices, which is why researchers examine the possibility. The key question is how well findings from limited experiments indicate what might happen on real devices and networks.
Both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
The experiments demonstrate that AI models can assist with parts of software replication across connected systems, including spreading to vulnerable cloud instances. Even if the tests used a custom setup and the process was slow, advocates of concern could argue that improving models and widely connected devices may make such capabilities more consequential over time, so early study and preparation are prudent.
The case against
The results fall well short of showing a practical threat to real-world devices: one model ran very slowly, the cloud tests used deliberately vulnerable instances, and the study did not test how systems would be breached. Critics could reasonably argue that presenting these constrained findings as evidence of an approaching botnet danger risks overstating what was demonstrated and fuelling fears beyond the available evidence.