Journal of Soft Computing Exploration
Vol. 7 No. 3 (2026): September 2026

Indonesian sign language (BISINDO) gesture detection using Yolov11-Pose

Achmad Dany Alfansyah (Department of Informatics, Telkom University Surabaya, Indonesia)
Ardian Yusuf Wicaksono (Department of Informatics, Telkom University Surabaya, Indonesia)
Pima Hani Safitri (Department of Informatics, Telkom University Surabaya, Indonesia)



Article Info

Publish Date
10 Aug 2026

Abstract

Sign language serves as the primary means of communication for deaf individuals, yet existing recognition systems for Bahasa Isyarat Indonesia (BISINDO) have not fully exploited keypoint-based pose estimation, particularly for dynamic alphabetic gestures. This study developed a BISINDO gesture detection system using the YOLOv11m-Pose algorithm integrated with MediaPipe hand landmark extraction, Letterbox image preprocessing, and a multi-phase class representation strategy that divided dynamic letters into two movement-phase classes. The main novelty of this study lies in the integration of YOLOv11-Pose with MediaPipe hand keypoint extraction and a multi-phase representation strategy for dynamic letters, which has not been previously applied to BISINDO alphabet detection. A dataset of 1.450 images across 29 classes was collected, augmented to 8.700 samples, and split into training, validation, and test sets. Evaluation on 870 test images yielded an overall accuracy of 99,43%, a macro precision of 99,44%, macro recall of 99,43%, and a mAP50 of 99,11%. All six dynamic letter classes achieved perfect prediction scores, confirming the effectiveness of the multi-phase representation approach. These results demonstrated that the proposed system was capable of reliable BISINDO alphabet detection and provided a solid foundation for further development toward full support for sign language communication.

Copyrights © 2026






Journal Info

Abbrev

journal

Publisher

Subject

Computer Science & IT Control & Systems Engineering Decision Sciences, Operations Research & Management Electrical & Electronics Engineering

Description

The journal focuses on publishing high-quality, original research and review articles in the field of Soft Computing, Informatics and Computer Science, emphasizing the development, application, and rigorous evaluation of Advanced Computational Methods, Artificial Intelligence (AI), Machine Learning ...