Internet memes in Indonesia are frequently weaponized to disseminate implicithate speech through sarcasm and cultural nuances. Automatically detectingsuch content is computationally challenging, and existing deep learningframeworks predominantly operate as opaque black boxes, lacking decisiontransparency. This study implements and optimizes a text-enriched dual-headmultitask learning architecture utilizing IndoBERTweet to concurrently classifyhatefulness and appropriateness within the INDOMEME dataset. Ratherthan processing raw image pixels, we employ a text-enrichment strategy wherevisual semantics are transcribed into textual descriptors via Optical CharacterRecognition and vision-language captioning. To bridge the interpretability gap,we deploy Local Interpretable Model-agnostic Explanations (LIME) to decodethe internal feature attributions of the architecture. Furthermore, advancedtraining optimizations, encompassing cosine annealing, gradient accumulation,class-weighted loss, and dynamic threshold calibration, were engineered toenhance model generalization. Experimental evaluations demonstrate thatthe optimized model achieves a Macro-F1 score of 0.812 for hatefulness and0.820 for appropriateness, surpassing the established baseline. Crucially, theLIME analysis unveils a pivotal finding: despite sharing an identical textualbackbone, the hate-specific head predominantly focuses on lexicons carryingsocial agitation, whereas the appropriateness head prioritizes general normviolations. These empirical findings substantiate that multitask learning enrichessemantic representation quality, offering a transparent framework fortrustworthy content moderation.
Copyrights © 2026