<![CDATA[Background: The surge of social media content in the modern era required an automated content moderation system to filter out hate speech and inappropriate content. Transformer-based models have been the main approach to this problem. However, it is important to ensure explainability and fairness, as the decision may not align with human reasoning and can create unintended bias. Objective: This study aims to investigate whether integrating transformer-based models with human-annotated rationales can drive the model to create better explainability and fairness in its output. Methods: We propose HAXE, a ranking-based attention-rational alignment framework that supervises attention with relative-importance constraints to prevent rationale distortion. Experiments were conducted on the HateXplain data using BERT, DistilBERT, and DeBERTa as the transformer models. Model performance is evaluated in the classification, explainability, and fairness domains, as well in the attention entropy analysis. Results: HAXE improves explainability across all models, as evidenced by the increased plausibility and faithfulness metrics score, with the largest gains observed in DistilBERT and DeBERTa. However, we observed that fairness effects are architecture-dependent, where DistilBERT and DeBERTa improved, whereas BERT showed degradation. Attention entropy analysis shows that BERT undergoes the largest entropy reduction, approximately 16%, compared with DistilBERT and DeBERTa, which is 10–13% greater sensitivity to attention collapse under the HAXE training objective. Conclusion: The results demonstrate that attention-rationale alignment strengthens explanation quality, but its fairness effects differ between model architectures. These trade-offs are required for supervision strategies that can adapt and be robust to different architectures. Future work should explore ER alignment, other supervision strategies, and evaluation across different datasets. Keywords: Explainable AI, Fairness in NLP, Hate Speech Detection, Attention Supervision, Transformer Models]]>
Copyrights © 2026
