Sign language detection using convolutional neural network for teaching and learning application

Wan Mohd Yaakob Wan Bejuri (Universiti Teknikal Malaysia Melaka)
Nur’Ain Najiha Zakaria (Universiti Teknikal Malaysia Melaka)
Mohd Murtadha Mohamad (Universiti Teknologi Malaysia)
Warusia Mohamed Yassin (Universiti Teknikal Malaysia Melaka)
Sharifah Sakinah Syed Ahmad (Universiti Teknikal Malaysia Melaka)
Ngo Hea Choon (Universiti Teknikal Malaysia Melaka)

Indonesian Journal of Electrical Engineering and Computer Science
Vol 28, No 1: October 2022

Teaching lower school mathematic could be easy for everyone. For teaching in the situation that cannot speak, using sign language is the answer especially someone that have infected with vocal cord infection or critical spasmodic dysphonia or maybe disable people. However, the situation could be difficult, when the sign language is not understandable by the audience. Thus, the purpose of this research is to design a sign language detection scheme for teaching and learning activity. In this research, the image of hand gestures from teacher or presenter will be taken by using a web camera for the system to anticipate and display the image's name. This proposed scheme will detects hand movements and convert it be meaningful information. As a result, it show the model can be the most consistent in term of accuracy and loss compared to others method. Furthermore, the proposed algorithm is expected to contribute the body of knowledge and the society.