Will My Refrigerator Destroy the Internet?

Smart Ass Take:

My refrigerator has one opinion, which it expresses by beeping when the door has been open for forty seconds, and until this week I considered that the outer limit of its ambition. Now Anthropic’s red team informs me that a cluster of fifteen off-brand TV boxes (about $55 each, the kind sold next to phone chargers at gas stations) can collectively run a reasonably smart AI model, and that small models may soon be able to copy themselves from gadget to gadget with no human involved. The good news is that this apocalypse would run at 0.39 tokens per second. Skynet, but it takes a day and a half to think of something. The researchers’ own analogy is a slime mold, which is both the most honest and the least comforting comparison I’ve read all year, because nobody has ever successfully reasoned with mold.

Article Excerpt:

Our experiments suggest that small AI models may be months away from being capable enough to self-replicate on commercial internet-of-things (IoT) devices, like smart ovens and off-brand Android TV boxes.

Article Summary:

Anthropic’s Frontier Red Team asked a question nobody wanted asked: could an AI model spread itself like a computer worm, carrying its own brain along and running it on whatever cheap, badly secured gadget it lands on? They wired together 15 bargain Android TV boxes and ran a 26-billion-parameter model across them, slowly but successfully. Smart fridges, ovens, vacuums and lawnmowers are named as potential hosts. Your appliances are, technically, a server farm.

The hard part for a would-be AI worm is setting up its own software on unfamiliar hardware. A 9B model managed it on 8 of 9 hardware types. In a simplified test network, only GLM-5.2 handled the full cycle (find a device, copy itself over, get running, repeat), reaching 8 of 8 nodes in 4 of 5 trials. Plotting how fast small models improve, the authors estimate a small model could hit that capability level by the end of 2026.

It would be slow. Scaled to the size of the Kimwolf botnet (2 million-plus devices), you’d get roughly 133,000 copies with the combined horsepower of about 40 datacenter GPUs, needing days per replication. Hence the slime mold comparison. Unsettlingly, copies that inherited notes from their predecessors improved, with success rates climbing from 45% to between 61% and 84%. Small models were bad at taking advice from a smarter model, which is relatable.

The defenses are mostly unglamorous: such worms would create odd network traffic, ISPs can quarantine infected customers (it cleared 92% of Mirai infections in two weeks), and manufacturers can stop shipping devices with default passwords. The authors’ conclusion: we’re seeing “signs of life,” and should prepare for these things to stick around.

Read the Full Article