Google DeepMind and a consortium of 33 other research institutions have initiated a project called "Open-X Embodiment" to address a significant challenge in robotics. The challenge involves the extensive effort required to train machine learning models for each unique robot, task, and environment. Typically, specialized models are needed for individual robots, making it challenging to adapt them to different scenarios. This project seeks to create a general-purpose AI system capable of working with various types of physical robots and performing multiple tasks.
To achieve this goal, the project introduces two key components:
-
A Diverse Dataset: The researchers collected data from 22 different robot embodiments at 20 institutions across various countries. This dataset includes examples of over 500 skills and 150,000 tasks across more than 1 million episodes. An episode represents a sequence of actions a robot takes when attempting a task.
-
Transformer-Based Models: The project utilizes deep learning models based on the transformer architecture, similar to those used in large language models. Specifically, there are two models—RT-1-X and RT-2-X. RT-1-X is built on Robotics Transformer 1 (RT-1), while RT-2-X is built on RT-1's successor, RT-2, which is capable of understanding natural language commands.
The results of testing RT-1-X and RT-2-X on various tasks and robots have been promising. RT-1-X achieved a 50% higher success rate compared to specialized models in tasks like picking and moving objects and opening doors. It also demonstrated the ability to generalize skills to different environments and types of robots. RT-2-X outperformed RT-2, particularly in tasks requiring spatial understanding.
The project's success suggests that models trained on a diverse set of examples can outperform specialized models in most tasks and can be applied to a wide range of robots.
The researchers have open-sourced the Open X-Embodiment dataset and a smaller version of the RT-1-X model. While these tools have the potential to transform the way robots are trained and accelerate research in the field of robotics, the team continues to explore future research directions, including combining these advances with insights from other models like RoboCat and investigating how different dataset mixtures may impact cross-embodiment generalization.
Overall, the project aims to make it easier and faster to train and deploy robots by creating AI systems that can adapt to various robots and tasks, reducing the need for specialized models for each scenario.
Comments