New AI Systems Revolutionize Robot Training with Virtual Simulations

Two innovative AI systems are using video and photos to create virtual simulations that train robots to function in real-world environments. These new technologies, developed by researchers at the University of Washington, have the potential to significantly reduce the costs of training robots to navigate complex settings like homes and workplaces.

Robots, which are already effective in structured environments such as assembly lines, struggle in dynamic spaces where objects and people constantly move. However, with the introduction of AI-powered simulations, robots can now virtually learn to handle real-life scenarios, improving their functionality without costly physical trials.

One of the systems, called RialTo, allows users to scan a space with a smartphone to generate a “digital twin” of the environment. A robot can then train itself in this virtual setting, practicing tasks such as opening drawers or maneuvering around obstacles. Meanwhile, another system, URDFormer, quickly creates hundreds of generic simulations using images from the internet, enabling large-scale training at a fraction of the cost.

Abhishek Gupta, a UW assistant professor and co-senior author on both studies, emphasizes the impact these systems could have: “The systems allow robots to be trained cheaply in simulations, helping them function safely and effectively in real spaces.”

The research was presented at the Robotics Science and Systems conference in Delft, Netherlands, and signals a promising future where AI-driven robots can seamlessly interact in our everyday lives.