Hybrid ViT–CNN Model for Automatic Monkeypox Skin Lesion Diagnosis - Triwerdaya, Aji; Utami, Ema (Journal of Electrical Engineering and Computer (JEECOM), 2025)
Monkeypox is a re-emerging zoonotic disease that presents with skin lesions resembling other dermatological conditions, which complicates reliable diagnosis. This study introduces a hybrid deep learning framework that integrates Vision Transformers (ViT) with Convolutional Neural Networks (CNN) for automatic classification of monkeypox lesions. Three hybrid scenarios were evaluated: ViT + DenseNet121, ViT + ResNet50, and ViT + InceptionV3.A combined dataset of PAD-UFES-20 and the Monkeypox Skin Lesion Dataset (MSLD), containing more than 2,500 dermoscopic images resized to 224×224 pixels, was used to train all models from scratch. Unlike prior works that relied on transfer learning and extensive augmentation, this study establishes a reproducible baseline without such enhancements. Model performance was assessed using accuracy, precision, recall, F1-score, and ROC-AUC, as well as computational efficiency metrics including training time and inference speed.The results show that hybrid ViT–CNN architectures achieved consistently better performance than single networks. Among the three scenarios, ViT + InceptionV3 provided the most balanced outcome, This approach combines reliable diagnostic accuracy with efficient inference. These findings demonstrate the value of integrating CNN-based local feature extraction with the global contextual modeling capacity of ViTs.This study establishes an experimental benchmark for monkeypox lesion classification and identifies hybrid architectures as a viable direction for future development. The framework can be extended with transfer learning, advanced augmentation, and lightweight optimization techniques, supporting potential deployment in resource-limited healthcare environments.