Startup taps video-game inputs to feed AI models learning real-world physics

Worldmodeldata, a UK-based startup backed by AI leaders such as Yann LeCun, is turning video-game controller inputs and visual streams into massive training sets for "world models" that aim to grasp real-world physics. The firm claims licensing of nearly 1 million hours of gameplay data, though it does not name the titles, and envisions future schemes to compensate gamers directly. Proponents argue that the sheer scale and diversity of game environments can supply the cause-and-effect data lacking in current AI corpora, potentially speeding up progress in robotics, autonomous vehicles and other embodied AI applications.

Critics, including Nvidia’s Ming-Yu Liu and University of Surrey’s Xiatian Zhu, caution that video-game physics are often coarse approximations, making them ill-suited for tasks requiring fine motor control, and suggest the data may be better for generating realistic virtual scenes. Venture-capital partner Nicole Fraenkel of Khosla Ventures stresses the need to capture corner cases, given the high cost of errors in real-world deployments. The concept remains experimental, with the AI community still assessing whether game-derived data can truly bridge the gap for world models.