Digitus : Journal of Computer Science Applications
Vol. 3 No. 4 (2025): October 2025

Evaluating Deep Learning Models for Humanitarian Sentiment Classification in Crisis Tweets: A Benchmark Study

Junaedi, Edi (Unknown)



Article Info

Publish Date
06 Oct 2025

Abstract

Social media platforms have emerged as essential channels for real time crisis communication, offering valuable insights into public sentiment and humanitarian needs during emergencies. This study benchmarks the performance of state of the art deep learning models for classifying sentiment and humanitarian relevance in crisis related tweets. Using publicly available datasets CrisisMMD, HumAID, and CrisisBench we evaluate three architectures: IDBO CNN BiLSTM, BERTweet, and CrisisTransformers. These models were assessed using cross validation and standard performance metrics (accuracy, F1 score, precision, and recall). Results indicate that CrisisTransformers outperform both traditional CNN LSTM hybrids and general purpose transformers, achieving an accuracy of 0.861 and F1 score of 0.847. Domain specific pretraining significantly enhances contextual understanding, particularly in multilingual and ambiguous tweet scenarios. While transformer models offer superior classification capabilities, their computational complexity poses challenges for real time deployment. Additionally, operational risks, such as data bias and misinformation, necessitate careful management through structured human oversight and the integration of explainable AI mechanisms. This research provides a robust comparison of NLP models for crisis applications and recommends strategies for effective deployment, including bias mitigation and fairness aware learning. The findings contribute to building ethical and efficient NLP systems for humanitarian response.

Copyrights © 2025






Journal Info

Abbrev

digitus

Publisher

Subject

Computer Science & IT Decision Sciences, Operations Research & Management

Description

Digitus : Journal of Computer Science Applications with ISSN Number 3031-3244 (Online) published by Indonesian Scientific Publication, is a leading peer-reviewed open-access journal. Since its establishment, Digitus has been dedicated to publishing high-quality research articles, technical papers, ...