Collaborative Robot and Deep Reinforcement Learning Method Using Facial Expression Feedback

Collaborative Robot Using Deep Reinforcement Learning with Facial Expression Feedback for Reward Optimization

This technology utilizes vision data from a humanoid robot to estimate the tilt of an object (a table) held by a user from state images, and determines and executes optimal balancing movements through a Deep Q-Network (DQN) model. It features a mechanism that enhances learning efficiency by analyzing the user's facial expressions in real-time to generate emotion-based feedback, which is then integrated with environmental reward values in the reinforcement learning model.

Conventional reward shaping methods are cumbersome, as they require humans to provide manual feedback via separate input devices. Furthermore, they are limited in feedback types and require a high level of specialized engineering expertise for agent modeling, which restricts the implementation of natural robot learning environments.

This technology captures images of the user's face, maps their emotions into a 2D emotional space using an expression evaluation model, and generates automated facial expression feedback based on this data. We propose an interactive deep reinforcement learning structure that combines this feedback value with environmental rewards using preset weights, then updates parameters to minimize the Q-function error of the DQN model.

‍

‍

Key Features:

• A camera that simultaneously captures state images of the held table and the user's evaluative facial expressions.
• A collaborative robot control module that inputs state images into a motion decision model to determine balancing actions and performs reinforcement learning by incorporating facial expression feedback.
• A collaborative robot drive module that physically operates the robot based on the determined balancing movements.
• A facial expression feedback output unit that estimates the user's emotions from evaluative facial images and outputs corresponding feedback values.

‍