International Journal of Advances in Intelligent Informatics
Vol 12, No 3 (2026): August 2026

Validation of a large language models-based socio-political sentiment analysis system with a retrieval-augmented generation approach

Sidharta Sidharta (Bina Nusantara University)
Hady Pranoto (Bina Nusantara University)
Frederik Gasa (Bina Nusantara University)
Nur Kholis (Bina Nusantara University)
Ardvin Kester Ong (School of Industrial Engineering and Engineering Management, Mapua University)



Article Info

Publish Date
31 Aug 2026

Abstract

Socio-political discourse on digital platforms generates large volumes of unstructured textual data, posing challenges for reliable sentiment analysis and contextual interpretation. Conventional sentiment analysis approaches often lack contextual grounding, while direct use of Large Language Models (LLMs) may introduce hallucinations, bias, and excessive generalization. To address these issues, this study proposes an LLM-based sentiment analysis system that integrates multi-source data acquisition, zero-shot sentiment classification, Retrieval-Augmented Generation (RAG), and generative reasoning for contextual interpretation. Unlike prior studies that mainly focus on improving model accuracy, this research emphasizes system-level validation of the entire analytical pipeline. The evaluation includes sentiment calibration using a Golden Dataset, data ingestion performance analysis, retrieval quality assessment based on semantic distance, and generative evaluation using an LLM-as-a-Judge framework. Experimental results indicate that the system provides stable data acquisition and produces coherent analytical reports. However, retrieval noise and neutrality bias introduced by LLM alignment mechanisms still affect the groundedness of generated outputs. These findings highlight the importance of retrieval quality and demonstrate the potential of LLMs as reasoning and evaluation components for analyzing complex socio-political discourse.

Copyrights © 2026






Journal Info

Abbrev

IJAIN

Publisher

Subject

Computer Science & IT

Description

International journal of advances in intelligent informatics (IJAIN) e-ISSN: 2442-6571 is a peer reviewed open-access journal published three times a year in English-language, provides scientists and engineers throughout the world for the exchange and dissemination of theoretical and ...