Precision-Driven Real-Time Pose Estimation for Therapeutic Interventions: Advanced Heatmap Regression, Reference Video Alignment, and Real-Time Corrective Feedback

Abstract

Accurate movement and posture are essential for effective physical therapy, as improper form can hinder recovery and worsen injuries. This project introduces a real-time human pose estimation system specifically designed for physical therapy, providing precise feedback on body alignment. Utilizing a mod- ified YOLOv8 architecture with custom heatmap regression, the system monitors key joints—particularly the wrist, elbow, and shoulder—vital for upper-body rehabilitation. Initially trained on a combined MPII and COCO 2017 dataset, the model was fine-tuned on a custom dataset of 6,000 images derived from 1,250 video frames under varied lighting conditions, with a 380% augmentation rate to improve robustness across scenarios. Achieving a detection accuracy of 91.61%, the system surpasses widely used models like OpenPose and MediaPipe, which deliver accuracies of 85% and 88%, respectively. With an average frame rate of 27.94 FPS and latency of 19.24 milliseconds per frame, the system provides instant feedback, enabling users to adjust posture in real time. Personalized guidance is offered by calculating the distance between live and reference keypoints, maintaining a mean keypoint detection error under 5 pixels. This real- time corrective feature enhances rehabilitation by empowering users to self-adjust and allowing healthcare providers to track progress effectively. By focusing on physical therapy-specific movements, this system represents a significant advancement in integrating AI-driven solutions into rehabilitation, enhancing both effectiveness and accessibility.