This technology is an AR-based robot arm control system that detects objects in real-time from video captured via an AR device and calculates their 3D coordinates using ray casting and mesh generation to precisely control the target position of a robot arm.
Conventional controller or eye-tracking methods are difficult for individuals with physical disabilities, such as quadriplegia, to operate directly. Furthermore, these methods present inconveniences and collision risks, as users must simultaneously monitor the screen and the robot arm's position.
This technology proposes a method where an AR device recognizes the user's gaze to select a specific object, calculates the relative distance and 3D coordinates between the object and the robot arm, and enables the robot arm to automatically move and perform grasping tasks, ensuring intuitive and safe operation. It can be utilized for rehabilitation assistance, support for daily living for people with disabilities, and remote operations, significantly improving the independence and quality of life for users with physical limitations.
This invention was developed with support from the Ministry of Science and ICT for the "Development of Customized Brain-Robot Interface for the Physically Disabled with Improved Accuracy, Speed, and Convenience" and the "Development of Non-invasive BCI Integrated Brain-Cognitive Computing SW Platform Technology for Controlling Real-life Appliances and AR/VR Devices via Thought" (BCI-General/Sub-project 1) projects.
N/A
