Jurnal Kecerdasan Buatan dan Teknologi Informasi
Vol. 5 No. 3 (2026): September 2026 In progress.

Interpreting Text-Enriched Dual-Head Multitask Learning for Indonesian Hateful Meme Detection Using Explainable AI

Selamet Riadi (Universitas Teknologi Mataram)
Emi Suryadi (Universitas Teknologi Mataram)
Muhamad Masjun Efendi (Universitas Teknologi Mataram)
Bahtiar Imran (Universitas Teknologi Mataram)
Muhammad Zamroni Uska (Universitas Hamzanwadi)



Article Info

Publish Date
01 Sep 2026

Abstract

Internet memes in Indonesia are frequently weaponized to disseminate implicithate speech through sarcasm and cultural nuances. Automatically detectingsuch content is computationally challenging, and existing deep learningframeworks predominantly operate as opaque black boxes, lacking decisiontransparency. This study implements and optimizes a text-enriched dual-headmultitask learning architecture utilizing IndoBERTweet to concurrently classifyhatefulness and appropriateness within the INDOMEME dataset. Ratherthan processing raw image pixels, we employ a text-enrichment strategy wherevisual semantics are transcribed into textual descriptors via Optical CharacterRecognition and vision-language captioning. To bridge the interpretability gap,we deploy Local Interpretable Model-agnostic Explanations (LIME) to decodethe internal feature attributions of the architecture. Furthermore, advancedtraining optimizations, encompassing cosine annealing, gradient accumulation,class-weighted loss, and dynamic threshold calibration, were engineered toenhance model generalization. Experimental evaluations demonstrate thatthe optimized model achieves a Macro-F1 score of 0.812 for hatefulness and0.820 for appropriateness, surpassing the established baseline. Crucially, theLIME analysis unveils a pivotal finding: despite sharing an identical textualbackbone, the hate-specific head predominantly focuses on lexicons carryingsocial agitation, whereas the appropriateness head prioritizes general normviolations. These empirical findings substantiate that multitask learning enrichessemantic representation quality, offering a transparent framework fortrustworthy content moderation.

Copyrights © 2026






Journal Info

Abbrev

JKBTI

Publisher

Subject

Computer Science & IT

Description

Jurnal Kecerdasan Buatan dan Teknologi Informasi or abbreviated JKBTI is a national journal published by the Ninety Media Publisher since 2022 with E-ISSN : 2964-2922 and P-ISSN : 2963-6191. JKBTI publishes articles on research results in the field of Artificial Intelligence and Information ...