To achieve success in the real world, walking robots must adapt to whatever surfaces they encounter, objects they carry, and conditions they are in, even if they’ve not been exposed to those conditions before. Moreover, to avoid falling and suffering damage, these adjustments must happen in fractions of a second.
Recently, a team of researchers from UC Berkeley, Carnegie Mellon University’s School of Computer Science, and Facebook AI announced Rapid Motor Adaptation (RMA), a breakthrough in artificial intelligence which enables legged robots to adapt in real-time to challenging, unfamiliar new surfaces, and circumstances. RMA utilizes a unique combination of two learned policies in simulation, a base policy trained through reinforcement learning and an adaptation module trained using supervised learning.
Until recently, legged robots have either been completely hand-coded for the environments they will inhabit or taught to explore their environments through a combination of hand-coding and learning techniques. RMA is the first wholly learning-based system to enable a legged robot to adapt to its environment from scratch by navigating and interacting with the world.
The tests demonstrate that RMA-enabled robots outperform alternative systems when walking over different slopes, obstacles, and surfaces and when given different payloads to carry. Since its ability is based entirely on what it encounters, an RMA-enabled robot can adjust to situations beyond the scope of consideration of programmers. RMA uses end-to-end learning throughout. It even directly outputs joint positions without relying on predefined leg motions or other control primitives.
However, few challenges occur when these skills are first learned in simulation and then practiced in the real world. The actual robot and its model in the simulator are generally different in small but significant ways.
Usually, an environment that is relatively standardized in simulation becomes much more diverse and complex in the real world, significantly when one factors the multitude of terrains in both indoor and outdoor spaces. Moreover, factors in the real world are never static, so one real-world environment that a legged robot can master can be completely different.
RMA overcomes these challenges by using two different subsystems, one is a base policy, and the other one is an adaptation module.
The base policy is learned in simulation using RL, using information about different environments. The researchers can’t simply deploy the robot with only this base policy because they don’t know the actual extrinsic it will encounter out in the real world.
Therefore, they rely on information that the robot teaches itself about its surroundings — information based on its most recent body movement. They use supervised learning to train the adaptation module to predict them from the recent history of the robot’s state. With the combination of a base policy and an adaptation module, the robot can adapt to new conditions in fractions of a second.
Once the RMA-enabled robot is deployed, the base policy and adaptation module work together and asynchronously. The base policy runs at a faster speed, whereas the adaptation module runs much slower. This enables the robot to perform robust and adaptive locomotion without any fine-tuning.
The experiments have demonstrated that the RMA-enabled robot successfully walks across several challenging environments, outperforming a non-RMA deployment and equaling if not bettering the hand-coded controllers used in a Unitree robot. The robot could walk on sand, mud, hiking trails, and over a dirt pile without a single failure in all the trials. It successfully walked down steps along a hiking trail in 70 percent of the tests.
The robot also successfully navigated a cement pile and a pile of pebbles in 80 percent of the tests, despite never seeing the unstable or sinking ground, obstructive vegetation, or steps during training. The team executed all the real-world deployments with the same policy without any simulation calibration or real-world fine-tuning. Moreover, it also maintained its height with a high success rate when moving with a 12 kg payload, which amounted to nearly 100 percent of its body weight.
RMA shows how advancements in AI can transform the field of robotics, enhancing the capabilities of robots while making those improvements more scalable to new conditions and applications. RMA-enabled robots could be deployed in various capacities, such as assistants in search and rescue operations, especially in areas that are too dangerous or impractical for human intervention. Moreover, RMA points the way to building AI systems that can adapt to many complex challenges in real-time by using data on the fly to understand the context in which a particular algorithm operates.
Source: https://ai.facebook.com/blog/ai-now-enables-robots-to-adapt-rapidly-to-changing-real-world-conditions/
Developers Page: https://ashish-kmr.github.io/rma-legged-robots/?fbclid=IwAR2hFpLMHWeUsQ1WT7qqXvpZ2p9diZb-rX70oPoO4rS7zA_819apBiQyHoU
Paper: https://ashish-kmr.github.io/rma-legged-robots/rma-locomotion-final.pdf
Supplementary Material: https://ashish-kmr.github.io/rma-legged-robots/rma-locomotion-supplementary.pdf
Consultant Intern: Kriti Maloo is currently pursuing her B.Tech from Indian Institute of Technology (IIT) Bhubaneswar. She is interested in Data Analytics and its applications in various domains. She is a Bibliophile and loves to explore new advancements in the field of technology.
