Article Per Year (5 Year)
p-Index From 2020 - 2025
0.947
P-Index
This Author published in this journals
All Journal Jurnal Ilmu Komputer dan Informasi Journal of ICT Research and Applications Indonesian Journal of Electrical Engineering and Computer Science
Erdefi Rakun
Universitas Indonesia
Computer Science & IT

Published : 5 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 5 Documents
Search

 1 
Improving Recognition of SIBI Gesture by Combining Skeleton and Hand Shape Features Erdefi Rakun; Noer FP Setyono
Jurnal Ilmu Komputer dan Informasi Vol. 15 No. 2 (2022): Jurnal Ilmu Komputer dan Informasi (Journal of Computer Science and Informatio
Publisher : Faculty of Computer Science - Universitas Indonesia

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.21609/jiki.v15i2.1014

Abstract

SIBI (Sign System for Indonesian Language) is an official sign language system used in school for hearing impairment students in Indonesia. This work uses the skeleton and hand shape features to classify SIBI gestures. In order to improve the performance of the gesture classification system, we tried to fuse the features in several different ways. The accuracy results achieved by the feature fusion methods are, in descending order of accuracy: 88.016%, when using sequence-feature-vector concatenation, 85.448% when using Conneau feature vector concatenation, 83.723% when using feature-vector concatenation, and 49.618% when using simple feature concatenation. The sequence-feature-vector concatenation techniques yield noticeably better results than those achieved using single features (82.849% with skeleton feature only, 55.530% for the hand shape feature only). The experiment results show that the combined features of the whole gesture sequence can better distinguish one gesture from another in SIBI than the combined features of each gesture frame. In addition to finding the best feature combination technique, this study also found the most suitable Recurrent Neural Network (RNN) model for recognizing SIBI. The models tested are 1-layer, 2-layer LSTM, and GRU. The experimental results show that the 2-layer bidirectional LSTM has the best performance.
Word recognition and automated epenthesis removal for Indonesian sign system sentence gestures Erdefi Rakun; I Gusti Bagus Hadi Widhinugraha; Noer Fitria Putra Setyono
Indonesian Journal of Electrical Engineering and Computer Science Vol 26, No 3: June 2022
Publisher : Institute of Advanced Engineering and Science

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.11591/ijeecs.v26.i3.pp1402-1414

Abstract

This research focuses on building a system to translate continuous Indonesian sign system (SIBI) gestures into text. In a continuous gesture, a signer will add an epenthesis (transitional) gesture, which is hand movement with no meaning but needed to connect the hand movement of one word with the next word in a continuous gesture. Reducing the number of irrelevant inputs to the model through automated epenthesis removal can improve the system's ability to recognize the words in continuous gestures. We implemented threshold conditional random fields (TCRF) to identify epenthesis gestures. The dataset consists of 2,255 videos representing 28 common sentences in SIBI. The translation system consists of MobileNetV2 as a feature extraction technique, removing epenthesis gestures found by the TCRF, and a long short-term memory (LSTM) for the classifier. With the MobileNetV2-TCRF-bidirectional LSTM model, the best word error rate (WER) and sentence accuracy (SAcc) were 33.4% and 16.2%, respectively. Intermediate-stage processing steps consisting of sandwiched majority voting of the TCRF and the removal of word labels whose number of frames is less than two frames, along with LSTM output grouping, were able to reduce WER from 33.4% to 3.4% and increase SAcc from 16.2% to 80.2%.
Translating SIBI (Sign System for Indonesian Gesture) Gesture-to-Text in Real-Time using a Mobile Device Misael Jonathan; Erdefi Rakun
Journal of ICT Research and Applications Vol. 16 No. 3 (2022)
Publisher : DRPM - ITB

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.5614/itbj.ict.res.appl.2022.16.3.5

Abstract

The SIBI gesture translation framework by Rakun was built using a series of machine learning technologies: MobileNetV2 for feature extraction, Conditional Random Field for finding the epenthesis movement frame, and Long Short-Term Memory for word classification. This high computational translation system was previously implemented on a personal computer system, which lacks portability and accessibility. This study implemented the system on a smartphone using an on-device inference method: the translation process is embedded into the smartphone to provide lower latency and zero data usage. The system was then improved using a parallel multi-inference method, which reduced the average translation time by 25%. The final mobile SIBI gesture-to-text translation system achieved a word accuracy of 90.560%, a sentence accuracy of 64%, and an average translation time of 20 seconds.
Recognizing Indonesian sign language (Bisindo) gesture in complex backgrounds Saputra, Muhammad Alfhi; Rakun, Erdefi
Indonesian Journal of Electrical Engineering and Computer Science Vol 36, No 3: December 2024
Publisher : Institute of Advanced Engineering and Science

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.11591/ijeecs.v36.i3.pp1583-1593

Abstract

Sign language, particularly Indonesian sign language (Bisindo), is vital for deaf individuals, but learning it is challenging. This study aims to develop an automated Bisindo recognition system suitable for diverse backgrounds. Previous research focused on greenscreen backgrounds and struggled with natural or complex backgrounds. To address this problem, the study proposes using Faster region-based convolutional neural networks (RCNN) and YOLOv5 for hand and face detection, MobileNetV2 for feature extraction, and long short-term memory (LSTM) for classification. The system is also designed to focus on computational efficiency. YOLOv5 model achieves the best result with a sentence accuracy (SAcc) of 49.29% and a word error rate (WER) of 16.42%, with a computational time of 0.0188 seconds, surpassing the baseline model. Additionally, the system achieved a SacreBLEU score of 67.77%, demonstrating its effectiveness in Bisindo recognition across various backgrounds. This research improves accessibility for deaf individuals by advancing automated sign language recognition technology.
End-to-end system for translating bahasa isyarat Indonesia sign language gestures into Indonesian text Putra, Satria; Rakun, Erdefi
Indonesian Journal of Electrical Engineering and Computer Science Vol 40, No 2: November 2025
Publisher : Institute of Advanced Engineering and Science

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.11591/ijeecs.v40.i2.pp719-734

Abstract

This study addresses critical challenges in developing an end-to-end bahasa isyarat Indonesia (BISINDO) SLT by integrating advanced deep learning techniques to overcome complex background interference, transitional gesture recognition, and limitations in dataset availability. While existing SLT systems struggle with isolated word recognition and manual preprocessing, our work introduces three key innovations: (1) implementation of YOLOv8 for optimized object detection, achieving 88% mAP and reducing WER to 11.40%, outperforming YOLOv5/v7 in handling complex backgrounds; (2) automated removal of transitional gestures using Threshold conditional random fields (TCRF), which attained 95.68% accuracy, significantly improving upon MobileNetV2’s performance (WER: 6.89% vs. 93.53%); and (3) end-to-end BISINDO SLT by expansion of the BISINDO dataset to 435 word labels, enabling comprehensive sentencelevel translation. Experimental results demonstrate the system’s robustness, with 8.31% of WER, 84.13% of SAcc, and 87.08% of SacreBLEU after dataset expansion and redundancy elimination through grouping methods. The proposed framework operates without manual intervention, marking a substantial advancement toward real-world applicability.
Co-Authors I Gusti Bagus Hadi Widhinugraha Misael Jonathan Noer Fitria Putra Setyono Noer FP Setyono Saputra, Muhammad Alfhi Satria Putra