Researchers at the University of Edinburgh in Scotland have now developed a new framework to train humanoid robots to walk like humans. The framework is designed with the help of human demonstrations. The framework makes use of two techniques – imitation learning and deep reinforcement learning. In order to attain natural and dynamic locomotion in these humanoid robots, above mentioned techniques are combined with theories of robotic control.
Video Credits: Yang et al.
Recently, researchers all over the world have been attempting to develop and evaluate techniques which will incorporate different locomotion styles in legged robots. One of the most common approaches used is imitation learning which means the robot must analyze demonstrations from the real world in order to walk like humans or animals. The details about the research are available in a paper pre published on arXiv.
One of the researchers who carried out this study, Chuanyu Yang said, “The key question we set out to investigate was how to incorporate (1) useful human knowledge in robot locomotion and (2) human motion capture data for imitation into deep reinforcement learning paradigm to advance the autonomous capabilities of legged robots more efficiently.”
While explaining the process Chuanyu Yang said that, “The key to replicating human-like locomotion styles is to introduce human walking data as an expert demonstration for the learning agent to imitate. Reward design is an important aspect of reinforcement learning, as it governs the behavior of the agent.”
The framework that is developed by Chuanyu Yang and his team is based on a unique reward design. It makes use of motion caption data of humans walking as a reference for training. It also makes use of two specialized hierarchical neural architectures – a Phase Functional Neural Network (PFNN) and a Mode Adaptive Neural Network (MANN).
The reward design consists of a task term and an imitation term. The task term offers necessary guidance for the humanoid robot in order to achieve high level locomotion. The imitation term enables the humanoid robot to achieve more human like and natural walking patterns.
Chuanyu Yang said, “By leveraging human walking motions as an expert demonstration for the artificial agent to imitate, we are able to speed up learning and improve overall task performance.”
While adding further they said, “Human demonstration knowledge allowed us to design our learning framework more meaningfully, which proves to be beneficial for motor skills and motor control in general.”
In order to evaluate the imitation learning framework, the researchers conducted a number of experiments in a simulated environment. They found out that it was capable of producing robust locomotion behaviours in a number of scenarios. It was able to do so even in the presence of disturbances or any undesirable factors.
The observations, conclusions and findings derived by the researchers through this study suggest that expert demonstrations can help to significantly enhance deep reinforcement learning techniques for the purpose of training robots on various locomotion styles.
Chuanyu Yang said, “In our future work, we also plan to extend the learning framework to imitate a more diverse and complex set of human motions, such as general motor skills across locomotion, manipulation and grasping.”
While adding further he said, “We also plan to research efficient simulation-to-reality policy transfer to enable fast deployment of the learned policies that adapt to real robots.”
