MediaPipe와 LSTM을 이용한 실시간 한국수어 학습 인식 시스템

초록

The need for sign language education to facilitate communication between the hearing-impaired and the general public is growing; however, limited access to professional institutions and the lack of real-time feedback hinder learning efficiency. This paper proposes a real-time Korean sign language recognition system that combines MediaPipe-based hand landmark extraction with an LSTM deep learning model. A dedicated dataset of 32 Korean sign language vocabularies was constructed under varied conditions, generating approximately 68,000 training sequences via a sliding window approach with an 80:20 train-test split. Experimental results demonstrate a recognition accuracy of 94.82% and a Marco F1-score of 0.95, with the LSTM model showing slightly higher accuracy and a more stable learning tendency compared to a GRU model. A cross-user evaluation achieved an average recognition rate of 83.3%, confirming practical generalization capability. The system is integrated with a web service to provide real-time feedback for improved accessibility in sign language education.

키워드