Claim Missing Document
Check
Articles

Found 4 Documents
Search

Hand Keypoint-Based CNN for SIBI Sign Language Recognition Handayani, Anik Nur; Amaliya, Sholikhatul; Akbar, Muhammad Iqbal; Wiryawan, Muhammad Zaki; Liang, Yeoh Wen; Kurniawan, Wendy Cahya
International Journal of Robotics and Control Systems Vol 5, No 2 (2025)
Publisher : Association for Scientific Computing Electronics and Engineering (ASCEE)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.31763/ijrcs.v5i2.1745

Abstract

SIBI is less widely adopted, and the lack of an efficient recognition system limits its accessibility. SIBI gestures often involve subtle hand movements and complex finger configurations, requiring precise feature extraction and classification techniques. This study addresses these issues using a Hand Keypoint-based Convolutional Neural Network (HK-CNN) for SIBI classification. The research utilizes Kinect 2.0 for precise data collection, enabling accurate hand keypoint detection and preprocessing. The optimal data acquisition distance between 50 and 60 cm from the camera is considered to obtain clear and detailed images. The methodology includes four key stages: data collection, preprocessing (keypoint extraction and image filtering), classification using HK-CNN with ResNet-50, EfficientNet, and InceptionV3, and performance evaluation. Experimental results demonstrate that EfficientNet achieves the highest accuracy of 99.1% in the 60:40 data split scenario, with superior precision and recall, making it ideal for real-time applications. ResNet-50 also performs well with 99.3% accuracy in the 20:80 split but requires longer computation time, while InceptionV3 is less efficient for real-time applications. Compared to traditional CNN methods, HK-CNN significantly enhances accuracy and efficiency. In conclusion, this study provides a robust and adaptable solution for SIBI recognition, facilitating inclusivity in education, public services, and workplace communication. Future research should expand dataset diversity and explore dynamic gesture recognition for further improvements.
Enhancing Semantic Similarity in Concept Maps Using LargeLanguage Models Wiryawan, Muhammad Zaki; Prasetya, Didik Dwi; Handayani, Anik Nur; Hirashima, Tsukasa; Pratama, Wahyu Styo; Putra, Lalu Ganda Rady
MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer Vol. 24 No. 3 (2025)
Publisher : Universitas Bumigora

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.30812/matrik.v24i3.4727

Abstract

This research uses advanced models, Generative Pre-trained Transformer-4 and Bidirectional Encoder Representations from Transformers, to generate embeddings that analyze semantic relationships in open-ended concept maps. The problem addressed is the challenge of accurately capturing complex relationships between concepts in concept maps, commonly used in educational settings, especially in relational database learning. These maps, created by students, involve numerous interconnected concepts, making them difficult for traditional models to analyze effectively. In this study, we compare two variants of the Artificial Intelligence model to evaluate their ability to generate semanticembeddings for a dataset consisting of 1,206 student-generated concepts and 616 link nodes (Mean Concept = 4, Standard Deviation = 4.73). These student-generated maps are compared with a reference map created by a teacher containing 50 concepts and 25 link nodes. The goal is to assess the models’ performance in capturing the relationships between concepts in an open-ended learning environment. The results show that demonstrate that Generative Pretrained Transformers outperform other models in generating more accurate semantic embeddings. Specifically, Generative Pre-trained Transformer achieves 92% accuracy, 96% precision, 96% recall, and 96% F1-score. This highlights the Generative Pretrained Transformer’s ability to handle the complexity of large, student-generatedconcept maps while avoiding overfitting, an issue observed with the Bidirectional Encoder Representationsfrom Transformer models. The key contribution of this research is the ability of two complex models and multi-faceted relationships among concepts with high precision. This makes it particularly valuable in educational environments, where precise semantic analysis of open-ended data is crucial, offering potential for enhancing concept map-based learning with scalable and accurate solutions.
Performance Evaluation of Artificial Intelligence Models for Classification in Concept Map Quality Assessment Pratama, Wahyu Styo; Prasetya, Didik Dwi; Widyaningtyas, Triyanna; Wiryawan, Muhammad Zaki; Putra, Lalu Ganda Rady; Hirashima, Tsukasa
MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer Vol. 24 No. 3 (2025)
Publisher : Universitas Bumigora

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.30812/matrik.v24i3.4729

Abstract

Open-ended concept maps generated by students give better flexibility and present a complex analysis process for teachers. We investigate the application of classification algorithms in assessing openended concept maps, with the purpose of providing assistance for teachers in evaluating student comprehension. The method used in this study is experimental methods, which consists of data collection, preprocessing, representation generation, and modelling with Feedforward Neural Network, Random Forest, Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Decision Tree, and Logistic Regression. Our dataset, derived from concept maps, consists of 3,759 words forming 690 propositions, scored carefully by experts to ensure high accuracy in the evaluation process. Results of this study indicate that K-NN outperformed all other models, achieving the highest accuracy and Receiver Operating Characteristic-Area Under the Curve scores, demonstrating its robustness in distinguishing between classes. Support Vector Machine excelled in precision, effectively minimizing false positives, while Random Forest showcased a balanced performance through its ensemble learning approach. Decision Tree and Linear Regression showed limitations in handling complex data patterns. FeedforwardNeural Network can model intricate relationships, but needs further optimization. This research concluded that Artificial Intelligence classification enables a better assessment for teachers, enables the path for personalized learning strategies in learning.
Mono Background and Multi Background Datasets Comparison Study for Indonesian Sign Language (SIBI) Letters Detection using YOLOv8 Andriyanto, Teguh; Handayani, Anik Nur; Ar Rosyid, Harits; Wiryawan, Muhammad Zaki; Azizah, Desi Fatkhi; Liang, Yeoh Wen
JOIV : International Journal on Informatics Visualization Vol 9, No 5 (2025)
Publisher : Society of Visual Informatics

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.62527/joiv.9.5.3462

Abstract

The research in this paper focuses on the detection of Indonesian Sign Language System (SIBI) letters using the YOLOv8 object detection model. The study compares two datasets, one with mono-background (a simple, uniform background) and another with multi-background (complex and varied backgrounds). The research aims to evaluate how the complexity of image backgrounds affects the performance of the YOLOv8 model in detecting SIBI letters This study uses a dataset consisting of 24 SIBI letters (excluding J and Z due to the complexity of their gestures), sourced from Mendeley. The dataset was processed with and without data augmentation (rotation, brightness adjustments, blur, and noise) to test the model under various conditions. Four models were trained and tested: one using mono-background images, another using augmented mono-background images, a third using multi-background images, and a final model trained on augmented multi-background images. The results showed that the YOLOv8 model performed best with the multi-background dataset, achieving a precision of 0.995, recall of 1.000, F1 score of 0.997, and mAP50 of 0.994Adding to the model made it better at generalizing, but it took longer to train. The study finds that multi-background datasets with augmentation make the model much better at finding SIBI letters in real-world settings. This makes it a promising tool for projects that aim to improve communication for deaf people in Indonesia. The study suggests that more research should be done on augmentation techniques and bigger datasets to make detection more accurate.