Technology news and discussion on TalkFuse.
The Next Evolution of AI Is Learning From Your Dodgy Gaming Skills
The AI industry is recognising that large language models, trained solely on text, are ill-equipped for physical-world tasks such as piloting autonomous vehicles or operating robots. Leading researchers, including Fei-Fei Li and Yann LeCun, are now focusing on "world models"—a different class of AI that requires training on visual and action data to understand real-world physics. However, unlike the vast text corpora available for language models, there is currently no comparable dataset for world models, creating a significant bottleneck to progress.
Worldmodeldata, a British startup advised by LeCun, aims to solve this problem by licensing video game data as training material. The company has secured approximately 1 million hours of gameplay footage from popular game studios, reasoning that games provide precisely what world models need: millions of diverse scenarios paired with player actions and consequences. Supporters argue that this abundance of diverse, varied data—including edge cases—makes video games ideal for training models on safety-critical tasks like autonomous vehicles and industrial robotics, though some, including Nvidia, remain sceptical of the approach.
In short
- AI researchers pivoting to "world models" requiring visual and action data for physical-world tasks
- Worldmodeldata licensing video game data; secured ~1 million hours from studios for training
- Video games provide cause-and-effect scenarios at scale unavailable elsewhere for model development
Steelman: both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
Video games offer a large, varied supply of visual scenes linked to actions and their consequences, which could help world models learn physical patterns and respond to unusual situations. Supporters argue that this data is far richer for action learning than text alone and could provide a practical starting point for robotics and autonomous systems where real-world data is scarce or costly.
The case against
Game footage may not capture the complexity, uncertainty or physical constraints of the real world, so models trained on it could learn patterns that fail to transfer to vehicles or industrial robots. Sceptics can reasonably argue that safety-critical systems need evidence from real environments and that the cost or difficulty of gathering suitable real-world data does not make simulated experience an adequate substitute.
Explained simply
Artificial intelligence systems like ChatGPT are trained on vast amounts of text, but they struggle with real-world physical tasks such as controlling self-driving cars or operating robots. Researchers including Fei-Fei Li and Yann LeCun are now focusing on a different approach called "world models", which learns by watching visual information paired with actions and what happens next.
World models require enormous amounts of training data showing how the physical world responds to different actions. Unlike text-based language models, which have access to vast libraries of written material, there is currently no comparable dataset for world models, creating a significant bottleneck.
A British startup called Worldmodeldata is attempting to solve this problem by using video game footage as training material. The company has secured approximately one million hours of gameplay from major game studios, reasoning that games provide exactly what world models need: diverse scenarios paired with player actions and their consequences. Some researchers see promise in this approach, particularly for safety-critical tasks, whilst others including the technology firm Nvidia remain sceptical.
AI Art Business Celebrity Companies Culture Entertainment Gaming Technology UK World