Hybrid Ensemble Retrieval-Augmented Generation for Indonesian Legal Consultation with Keyword Boosting - Suharyadi; Saputra, Irwansyah (Journal of Novel Engineering Science and Technology, 2025)
This study presents the design and evaluation of a fully local, hybrid ensemble Retrieval-Augmented Generation (RAG) system tailored for Indonesian legal consultation. By integrating sparse (BM25), dense (FAISS), and keyword-aware retrieval mechanisms, the system balances lexical, semantic, and domain-specific relevance to retrieve high-quality legal context. A curated dataset of 8,450 legal consultation articles was scraped from a trusted legal platform, cleaned through multi-stage pre-processing, and indexed for efficient retrieval. Retrieved documents are formatted into structured prompts and fed into locally hosted large language models (LLMs) using Ollama, allowing for complete offline operation. Experiments comparing five retrieval configurations TF-IDF, BM25, FAISS, ensemble BM25+FAISS, and ensemble with keyword boosting demonstrate that the hybrid ensemble with keyword boosting yields the most relevant and grounded answers. Both quantitative (retrieval score analysis) and qualitative (manual relevance rating) evaluations were performed, confirming the effectiveness of the ensemble strategy in improving answer quality. Additionally, the system achieves practical response times (12–20 seconds) on consumer-grade hardware without reliance on cloud services. This work makes a novel contribution by demonstrating that a hybrid ensemble retrieval framework, specifically tuned to the linguistic characteristics and retrieval challenges of Indonesian legal texts, can significantly enhance the performance of local RAG-based legal QA systems. Future directions include real-time indexing, fine-tuning of legal-domain LLMs, and extending the system to support other legal domains such as statutory law, regulations, and court rulings.