Method and Service Device for Outputting Gait Motion Information Using Reinforcement Learning Models

Reinforcement Learning-Based Gait Generation Technology Using Reward Functions Without Reference Motions

This technology combines deep reinforcement learning with finite state machines to generate character gait motions in real-time. It inputs dynamic states and character-specific parameters into a neural network to generate action information, learning natural gait policies through reward functions.

Conventional finite state machine-based control methods have limitations in achieving natural motion, while existing deep learning approaches require separate reference motion data for gait decision-making, reducing their versatility and efficiency.

By incorporating the minimization of gait parameter deviation, vertical axis maintenance, directional alignment, and joint torque minimization into the reward function, this technology determines optimal stance hip torque and joint angles without the need for reference motion data. It can be applied to bipedal robot control and the generation of character motions in games and animation, enabling natural gait implementation without the burden of data collection.

‍

‍

Key Features:

• A step of a computing device acquiring gait motion state information of a character at time t
• A step of inputting the acquired state information into a pre-built neural network model to generate action information
• A step of transmitting the action information at time t to the character or a device that outputs the character's motion
• A configuration where the neural network model receives gait motion states as input to determine action information that maximizes reinforcement learning rewards

‍

This invention was developed with support from the Ministry of Science and ICT for the development of biomechanical model-based intelligent control technology for human movement involving multi-level interactions, and the DeepXR: Deep Hyper-Reality research project.