Real-Time Recognition of American Sign Language Using Mediapipe and Machine Learning Techniques

Abstract

Hand sign recognition has become an important area of study in computer vision and pattern recognition, with applications including sign language translation and human-computer interaction developing humanoid robots in medical applications, corporations and restaurants in some countries. Users can interact with the devices without actually touching them through hand sign recognition. This paper offers a comparative analysis of Machine Learning algorithms for American hand-sign recognition with a bespoke coordinate’s dataset including 87,000 photos, each measuring 200x200 pixels. The dataset has 26 classes, each representing a letter of the English alphabet, with 3000 images per class. In the pre-processing stage, MediaPipe, which is based on Convolutional Neural Netwok (CNN), extracts 21 points that represent the angles and directions of the hand for each image. Five models—Logistic Regression, K-Nearest Neighbors, Support Vector Machine, Decision Trees, and Random Forest—are evaluated for how well they can recognize the corresponding letter of each hand gesture. The results provide information on how well each method works for identifying American hand signs. The study shows that the Random Forest Algorithm works better than the others, achieving an accuracy of 98.83%. These findings are significant for developing automated systems that can use American Sign Language (ASL) for communication, such as sign language interpreters, smart home devices, and interactions with robots.