Engineering Science Letter
Vol. 5 No. 02 (2026): In Press - Engineering Science Letter

Multi-Modal Deep Learning Approach for Waste Management: Integrating Image Classification and Text Mining for Environmental Awareness

Santi Prayudani (Politeknik Negeri Medan)
Ainul Hizriadi (Universitas Sumatera Utara)
Yuyun Yusnida Lase (Politeknik Negeri Medan)



Article Info

Publish Date
27 Jun 2026

Abstract

Environmental degradation caused by inefficient waste management remains a major global challenge, largely due to the limitations of conventional systems that rely on manual waste sorting and limited utilization of heterogeneous data sources. This study proposes a novel multi-modal deep learning framework that integrates visual and textual information to enhance waste classification performance while simultaneously providing insights into environmental awareness. The proposed framework combines convolutional neural networks (CNNs) for waste image classification and a recurrent neural network with long short-term memory (LSTM) architecture for text analysis. Visual and textual feature representations are integrated through a feature-level fusion strategy using vector concatenation before final classification. The image dataset consists of six waste categories, cardboard, glass, metal, paper, plastic, and trash, while the textual dataset contains waste management descriptions, community feedback, and environmental discourse collected from public and field sources. Environmental awareness was assessed through text mining by identifying dominant themes related to recycling practices, waste sorting behavior, environmental responsibility, and public concern regarding pollution and sustainability issues. Experimental results demonstrate that the proposed multimodal framework achieves an accuracy of 88.9% and an F1-score of 0.89, outperforming image-only and text-only models with accuracies of 78.4% and 81.2%, respectively. This corresponds to absolute performance improvements of 10.5% over the image-based model and 7.7% over the text-based model, while reducing the classification error rate by 40.96%. Furthermore, the multimodal model exhibits superior robustness under degraded data conditions, with only a 4.7% reduction in accuracy compared to larger performance declines observed in unimodal approaches. The main contribution of this study lies in the integration of waste image recognition and environmental-awareness extraction within a unified multimodal learning framework, enabling not only accurate waste categorization but also the generation of behavioral and sustainability-related insights that support more intelligent and sustainable waste management systems.

Copyrights © 2026






Journal Info

Abbrev

ESL

Publisher

Subject

Computer Science & IT Control & Systems Engineering Engineering Industrial & Manufacturing Engineering Materials Science & Nanotechnology

Description

Engineering Science Letter is an international peer-reviewed letter that welcomes short original research submissions on any branch of engineering, computer science, and technology, as well as their applications in industry, education, health, business, and other fields. Artificial intelligence, ...