Finger Recognition Detection System Using Mediapipe as Communication Solution for People with Disabilities
Repository Politeknik Negeri Batam
Date
2024-12-25
Authors
M.WAHYU ELFANDER M.WAHYU ELFANDER
Journal Title
Journal ISSN
Volume Title
Publisher
Atlantis Press
Abstract
Sign language is a language used by people with disabilities, especially the deaf and mute to communicate. The problem is, not everyone can understand sign language. This study aims to create a system that can translate sign language. The system can also produce sound with the text to voice method. The system is built using Mediapipe to detect fingers that form sign language. The system performs classification by combining 2 machine learning models with the Artificial Neural Network (ANN) method. The first model is used to classify letters A-Z and the second model is used to classify movement patterns in letters J and Z. The accuracy of the first model is 94% and the accuracy of the second model is 95%. The model will also be evaluated using the confusion matrix technique to find recall, f1-score, and precision of 25 letters or classes.
Description
Keywords
HUMANITIES and RELIGION::Languages and linguistics::Sign language, confusion matrix, artificial neural network, machine learning
Citation
W. A. R. Wan Ali, A. Abdul Kassim, and S. M. Mohd Rashid, ‘Assistive Technology for Persons with Physical Disabilities: A Literature Review’, International Journal of Academic Research in Business and Social Sciences, vol. 14, no. 2, Feb. 2024, doi: 10.6007/ijarbss/v14-i2/20823. 2. R. Sandhiya, & V. Saranya. (2021). An application for mute and hearing impaired person to attend phone call. Scientific Hub of Applied Research in Engineering & Information Technology, 1(2), 19–22. https://doi.org/10.53659/shareit/2021/11 3. J. D. Bonvillian, K. Lee, T. T. Dooley, and F. T. Loncke, Simplified Signs A Manual Sign Communication System for Special Populations Volume 1: Principles, Background, and Application, vol. 1. Open Book Publishers, 2020. doi: 10.11647/OBP.0205. 4. R. S. Fauzi, B. Irmawati, and N. Agitha, ‘KADARING SIBI (Indonesian Sign System Online Dictionary): Web-based Indonesian Sign System Learning App’, in Proceedings of the First Mandalika International Multi-Conference on Science and Engineering