Multimodal Transfer Learning for Anti-Inflammatory Medicinal Plant Leaf Classification using ResNet50 - Umniy Salamah; Nur Ani; Yuwan Jumaryadi; Agustiawan (LPPM Universitas Islam 45 Bekasi, 2026)
This study aimed to develop an AI-based image classification model using transfer learning methods to identify seven types of anti-inflammatory plant leaves commonly used in traditional medicine. The novelty of this research lies in approach to integrating Canny Edge detection and Gamma Correction with the ResNet50 architecture for multimodal fusion. The class plants, including Aloe Vera, Annona Muricata, Centella Asiatica, Muntingia Calabura, and Ocimum Basilicum, are known for their therapeutic properties and bioactive compounds. A dataset consisting of 350 images per species was collected, with images divided into training (70%), validation (20%), and testing (10%) sets. Data augmentation techniques such as rotation, flipping, and zooming were applied to improve model generalization. To enhance classification performance, pre-trained convolutional neural network (CNN) models, including ResNet50 and VGG16, were employed for transfer learning. The study also integrated image processing techniques, such as the Laplacian Filter, Canny Edge, and Gamma Correction, to extract additional features and improve the model’s accuracy. Among the different configurations tested, the combination of Canny Edge and Gamma Correction with ResNet50 yielded the best results, achieving a training accuracy of 89.3%, validation accuracy of 88.1%, and test accuracy of 87.0%. In contrast, the use of Laplacian Filter and Canny Edge with ResNet50 led to lower performance, suggesting that multimodal fusion of certain feature extraction methods could enhance classification accuracy. This research highlighted the potential of AI-driven approaches in the classification of medicinal plant leaves and offered a more efficient, accurate method for identifying anti-inflammatory plants used in traditional medicine.