Robot training Hybrid AI teaches robots to walk

Related Vendors

FAULHABER Drive Systems

Wuxi InfiMotion Technology Co., Ltd.

The DFKI in Bremen, Germany, has developed innovative control methods for complex systems (such as robots) that combine the advantages of fast learning with the reliability of mathematical verification.

With advances in the field of subsymbolic artificial intelligence, such as deep learning, the limitations of data-based methods in terms of safety and reliability are becoming increasingly clear, as experts from the German Research Center for Artificial Intelligence (DFKI) say. Because decisions made by machine learning are based neither on symbolic calculations nor are they explainable by logical rules – and therefore not mathematically provable. Specifically in safety-critical areas such as humanoid robotics, verifiability is however crucial to minimize the risk of malfunctions and to ensure a safe environment, it continues.

Reward functions help robots make decisions

Through symbolic specifications, such as a simple language for describing robot behavior, the project team has finally been able to create abstract kinematic models that can be mathematically verified. These abstractions define reward functions for reinforcement learning and allow the robot to verify its decisions based on the models. This improves the reliability of the decisions, ensuring stable and predictable robot movements and minimizing the risk of malfunctions or unexpected actions. In addition, it states, the desired robot behavior was modeled as a hybrid automaton – a mathematical model that describes both continuous and discrete behavior. This reduces the state space of the system, enabling more efficient learning, according to DFKI researchers.

A humanoid robot is running away from all the others

In addition, as part of the project, it was possible to achieve dynamic walking with the humanoid robot "RH5" of DFKI. To achieve this, the method of the zero moment point (the point on the robot's standing surface where the resulting ground force does not generate a tilting moment) was combined with the whole-body control approach. This combination maximizes the stability and performance of position-controlled robots. Thus, dynamic walking at different speeds and stride lengths could be made robust while effectively exploiting system boundaries in terms of both speed and range of motion. To the knowledge of the Bremen researchers, this is the first time that a humanoid robot has walked dynamically at up to 0.43 meters per second. Apart from systems with active toe joints, the RH5 is one of the fastest humanoid robots of similar size and drive modalities. To further improve the behavior of the RH5, additional algorithms for simulation and optimal control, which are based on the symbolic model of the system, were used.

Foundation for safe operation of humanoid robots is established

Because the precise modeling and optimization of movement sequences both increases the safety and efficiency of robots, the hybrid AI approach developed in the project can be seen as a blueprint for how the generation of reward functions from symbolic, logic-based AI can work. This is particularly relevant for applications where the systems or their malfunction pose a potential danger to humans, as the Bremen experts finally note.

The "VeryHuman" project was funded by the Federal Ministry of Education and Research (BMBF) from June 2020 to May 2024 under the funding code 01IW20004.