Development of user-customized online teaching technology based on GPT, enhanced by generative AI-based motion recognition

Abstract

Abstract This paper addresses current issues in existing joint tracking and motion recognition algorithms for Human Pose Estimation. It proposes a solution using the Numerical Discriminator Generative Adversarial Network (ND_GAN) to improve the performance of vision-based motion recognition technology. Existing algorithms face challenges in accurately tracking joints in crowded spaces or with users wearing special attire, resulting in reduced accuracy and inconsistent results. The proposed ND_GAN consists of three integrated modules, enabling more precise joint estimation even in complex environments. Experiments were conducted using yoga videos from Hanchoom, a home training platform by TDI, comparing the performance of MediaPipe and ND_GAN on both original and leg-masked videos. Quantitative evaluation showed that ND_GAN achieved 94.2% PCK@0.5, 93.1% PCK@0.2, and an F1-score of 92.8%, marking an improvement of over 30% compared to the existing MediaPipe model. Notably, ND_GAN consistently estimated joint coordinates even in occluded video conditions, demonstrating significant performance gains in scenarios where traditional models failed. Furthermore, ND_GAN outperformed cutting-edge state-of-the-art models such as DWPose and ViTPose by more than 15% in accuracy, confirming its robustness in real-world environments with various occlusion conditions. Additionally, a teaching system using GPT is introduced to provide real-time feedback on movement errors, supporting users in effectively learning the motions. This integrated approach marks a significant advancement in addressing Human Pose Estimation complexities, enhancing overall vision-based motion recognition efficacy.