<![CDATA[This study discusses the issue of customer service in export companies, which is still limited in terms of service availability and the ability to handle recurring inquiries from international customers. Limitations in human resources result in responses to customer inquiries not being performed optimally, especially outside of operating hours. This study aims to design, implement, and evaluate a Retrieval-Augmented Generation (RAG)-based chatbot to support the automation of customer service in export companies using the Botika Agentic Platform. The study contributes by implementing and quantitatively evaluating a RAG-based chatbot on a Platform as a Service (PaaS) environment using the DeepEval framework with the LLM-as-a-Judge approach. The methods used in this study include four stages, namely data collection through interviews and analysis of company documents, system design, implementation through the Botika platform, and testing using the DeepEval framework with the LLM-as-a-Judge approach. The testing dataset consists of 93 questions covering four knowledge base domains, namely company profile, export and shipping procedures, partnership information, and product information. Evaluation results indicate that the chatbot achieved an average Answer Relevancy score of 0.98, Faithfulness of 1.00, and Contextual Relevancy of 0.88. The system performs well, with high Answer Relevancy and Faithfulness scores, while Contextual Relevancy is relatively lower, particularly in the product information domain. This is due to the dense structure of product documents, which contain many product variations within a single document, thereby affecting the accuracy of the retrieval process. Overall, the proposed RAG-based chatbot demonstrates its potential to support customer service automation in export companies by providing accurate and contextually grounded responses, while highlighting the importance of knowledge base organization for improving retrieval performance.]]>
Copyrights © 2026
