<![CDATA[This study proposes and evaluates a Retrieval-Augmented Generation (RAG) system integrating ChatGPT with FastText word embeddings for processing Indonesian-language PDF documents. The limitations of Large Language Models (LLMs) in handling domain-specific queries motivated the development of an RAG architecture with an advanced retrieval strategy. The CRISP-DM methodology was applied to compare two models: (1) a Baseline Model using pure FastText cosine similarity search, and (2) a Proposed Model combining Hybrid Search (FastText + BM25 with dynamic weighting), Maximal Marginal Relevance (MMR), and Cross-Encoder TF-IDF re-ranking. Evaluation was conducted using the RAGAS framework with five metrics—Context Relevance, Context Precision, Context Recall, Faithfulness, and Answer Relevance—across 20 test queries on the PT Omni Hottilier Booking Engine user guide. Results show that the Proposed Model outperformed the Baseline on 16 of 20 queries (80%), with the Overall Score increasing from 0.7853 to 0.8572 (+9.16%). The most significant improvements were observed in Context Relevance (+19.85%), Context Recall (+16.76%), and Faithfulness (+14.76%). The developed architecture is proven effective as an adaptive solution for transforming technical documents into intelligent chatbot knowledge bases, particularly for the morphologically complex Indonesian language.]]>
Copyrights © 2026
