A developer known as SlvDev has demonstrated that even a large language model can run on an ESP32-S3 microcontroller costing less than $10, a device normally used for basic sensors and IoT tasks rather than generative AI. The feat, documented on GitHub and showcased on the Better Stack YouTube channel, shows how aggressive memory-saving techniques can squeeze AI workloads onto hardware far too small and cheap to handle them by conventional means.

To achieve this, SlvDev used TinyStories, a 28.9-million-parameter model from Microsoft Research, roughly 10,000 times smaller than modern frontier models. Quantisation cut the model's memory footprint from 60MB to 14.9MB, and a technique borrowed from Google's Gemma models, called per-layer embedding, offloaded around 12MB of weights to flash storage, leaving only about 2MB needed in active memory. The result was a working chatbot-style model running at 9.88 tokens per second on the ESP32-S3's 520KB of SRAM and 8MB of PSRAM, faster than the average person can read.