Article Per Year (5 Year)
p-Index From 2021 - 2026
0.444
P-Index
This Author published in this journals
All Journal Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) Journal of Applied Data Sciences
Nova Agustina
Universitas AMIKOM Yogyakarta
Computer Science & IT Control & Systems Engineering Decision Sciences, Operations Research & Management Engineering

Published : 2 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 2 Documents
Search

 1 
Comparison of Multilingual Model Sensitivity for Political Fact Verification with Integrated Multi-Evidence Nova Agustina; Kusrini Kusrini; Ema Utami; Tonny Hidayat
Journal of Applied Data Sciences Vol 7, No 2: May 2026
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v7i2.1198

Abstract

Political news is frequently targeted by the dissemination of fake news on social media, which can influence public opinion and undermine trust in democratic processes. The main challenge in addressing this issue lies in the limited sensitivity of cross-lingual fact verification models in capturing semantic relationships between claims and evidence in long-text, multi-evidence settings. Existing approaches often struggle to assess the relevance and quality of evidence, resulting in suboptimal verification performance. This study compares three multilingual Large Language Models (LLMs), namely mBERT, XLM-R, and LaBSE, for political fact verification using an integrated multi-evidence approach. Experiments are conducted on the PolitiFact dataset, with performance evaluated using sensitivity, accuracy, precision, and F1-score metrics.The results indicate that mBERT achieves the highest overall sensitivity at 89.44%, followed by LaBSE at 81.81% and XLM-R at 78.81%. However, mBERT exhibits lower precision, whereas LaBSE provides a better balance between precision (87.02%) and accuracy (86.46%), resulting in an F1-score of 84.33%. XLM-R demonstrates lower sensitivity but maintains competitive precision (85.47%) and accuracy (84.60%), with an F1-score of 82.00%. Sensitivity analysis based on the number of evidence reveals distinct model behaviors, where mBERT performs optimally with six pieces of evidence, XLM-R is more effective under limited evidence conditions, and LaBSE shows a stable and increasing sensitivity trend as the amount of evidence increases, indicating robustness in multi-evidence scenarios. Further statistical analysis shows that XLM-R has the lowest performance variance, while LaBSE statistically outperforms mBERT in several evaluation aspects. Overall, LaBSE is recommended as the most balanced model for multi-evidence-based political fact verification.
DiG-MFV: Dual-integrated Graph for Multilingual Fact Verification Nova Agustina; Kusrini; Ema Utami; Tonny Hidayat
Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) Vol 9 No 4 (2025): August 2025
Publisher : Ikatan Ahli Informatika Indonesia (IAII)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29207/resti.v9i4.6695

Abstract

The proliferation of misinformation in political domains, especially across multilingual platforms, presents a major challenge to maintaining public information integrity. Existing models often fail to effectively verify claims when the evidence spans multiple languages and lacks a structured format. To address this issue, this study proposes a novel architecture called Dual-integrated Graph for Multilingual Fact Verification (DiG-MFV), which combines semantic representations from multilingual language models (i.e., mBERT, XLM-R, and LaBSE) with two graph-based components: an evidence graph and a semantic fusion graph. These components are processed through a dual-path architecture that integrates the outputs from a text encoder and a graph encoder, enabling deeper semantic alignment and cross-evidence reasoning. The PolitiFact dataset was used as the source of claims and evidence. The model was evaluated by using a data split of 70% for training, 20% for validation, and 10% for testing. The training process employed the AdamW optimizer, cross-entropy loss, and regularization techniques, including dropout and early stopping based on the F1-score. The evaluation results show that DiG-MFV with LaBSE achieved an accuracy of 85.80% and an F1-score of 85.70%, outperforming the mBERT and XLM-R variants, and proved to be more effective than the DGMFP baseline model (76.1% accuracy). The model also demonstrated stable convergence during training, indicating its robustness in cross-lingual political fact verification tasks. These findings encourage further exploration in graph-based multilingual fact verification systems.
Co-Authors Ema Utami Kusrini TONNY HIDAYAT