Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI)
Vol. 15 No. 2 (2026)

Static Sign Language Classification Using RGB, Skeleton Image, and Fusion Representations: A Cross-Dataset Evaluation

Moch. Iskandar Riansyah (Department of Electrical Engineering, Telkom University, Surabaya Campus, Surabaya, Indonesia)
Mohamad Yani (Department of Computer Engineering, Telkom University, Surabaya Campus, Surabaya, Indonesia)
Ubaidillah Umar (Department of Computer Engineering, Telkom University, Surabaya Campus, Surabaya, Indonesia)
Helmy Widyantara (Department of Information Technology, Telkom University, Surabaya Campus, Surabaya, Indonesia)



Article Info

Publish Date
31 Jul 2026

Abstract

This study investigates static sign language classification by examining three image-based input representations, namely RGB images, skeleton images, and RGB+skeleton fusion, using two datasets with different visual characteristics. The first dataset reflects a relatively controlled acquisition setting, whereas the second dataset contains more complex background and visual variations. As an initial baseline, eight pretrained convolutional neural network (CNN) architectures were evaluated, and two representative models were subsequently selected for more detailed analysis. The baseline evaluation indicates that ResNet50 and EfficientNetB0 achieve the most competitive performance when RGB images are used as input. Further analysis of input representations shows that skeleton images are highly effective, particularly on the more challenging dataset, while RGB+skeleton fusion does not consistently improve classification performance. The cross-dataset evaluation further reveals a considerable performance drop across all configurations, suggesting the presence of a strong domain shift between the two datasets. In the A-B scenario, EfficientNetB0 with RGB input yields the best results, while in the B-A scenario, EfficientNetB0 with skeleton input shows the most stable performance. These findings indicate that the most effective input representation depends on the direction of domain transfer and that high intra-dataset performance does not necessarily reflect good generalization capability.

Copyrights © 2026






Journal Info

Abbrev

janapati

Publisher

Subject

Computer Science & IT Education Engineering

Description

Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI) is a collection of scientific articles in the field of Informatics / ICT Education widely and the field of Information Technology, published and managed by Jurusan Pendidikan Teknik Informatika, Fakultas Teknik dan Kejuruan, Universitas ...