Google Launches Gemini Robotics On-Device: AI for Bi-Arm Robots

Google DeepMind has announced the launch of Gemini Robotics On-Device, a groundbreaking artificial intelligence model designed specifically for robotics. This advanced model operates entirely on a local device, eliminating the need for an active data connection. It is tailored to enable bi-arm robots to perform complex tasks in real-world environments by integrating voice, language, and action (VLA) processing.

Carolina Parada, Senior Director and Head of Robotics at Google DeepMind, highlighted the model's low-latency performance and flexibility in a recent blog post. By operating independently of the cloud, the model is particularly well-suited for latency-sensitive environments and real-time applications where constant internet connectivity is not feasible.

Currently, access to Gemini Robotics On-Device is restricted to participants in Google’s trusted tester program. Developers can experiment with the AI system through the Gemini Robotics software development kit (SDK) and the company's MuJoCo physics simulator.

Although specific details about the model’s architecture or training methodology have not been disclosed, Google has outlined its robust capabilities. Designed for bi-arm robotic platforms, the system requires minimal computing resources. It can adapt to new tasks using only 50 to 100 demonstrations, significantly accelerating deployment in diverse settings.

In internal trials, the model demonstrated the ability to interpret natural language commands and perform a wide array of sophisticated tasks. These include folding clothes, unzipping bags, and handling unfamiliar objects. It also successfully completed precision tasks such as those found in industrial belt assembly, showcasing high levels of dexterity.

Originally trained on ALOHA robotic systems, Gemini Robotics On-Device has been adapted to work with other bi-arm robots, including Franka Emika’s FR3 and Apptronik’s Apollo humanoid robot. The model exhibited consistent generalization performance across different platforms, even when faced with out-of-distribution tasks or multi-step instructions.

This development marks a significant step forward in the field of robotics, offering the potential for more autonomous and versatile robots in various industries. The ability to operate without internet connectivity and the adaptability to new tasks make Gemini Robotics On-Device a promising tool for enhancing productivity and efficiency in real-world applications.