Implementation of an Academic Information Chatbot Based on Retrieval-Augmented Generation (RAG) Using LLaMA 3.1

Mochamad Alfan Rosid (Muhammadiyah University of Sidoarjo, Indonesia)
Muhammad Saddam Heykal Bustomy (Informatics Study Program, University of Muhammadiyah Sidoarjo, Indonesia)
Suprianto (Informatics Study Program, University of Muhammadiyah Sidoarjo, Indonesia)
Hamzah Setiawan (Informatics Study Program, University of Muhammadiyah Sidoarjo, Indonesia)

IJHCS
Vol. 8 No. 1 (2026): International Journal of Human Computing Studies (IJHCS)

Providing fast and accurate academic information is a fundamental requirement for universities. However, users often experience difficulties in efficiently searching for specific information on study program websites. This study aims to develop a Retrieval-Augmented Generation (RAG)-based academic information chatbot that is directly integrated with the Informatics Study Program website at Muhammadiyah University of Sidoarjo. The system was built using the LangGraph architecture, ChromaDB vector database, and LLaMA 3.1 model through the Groq API. Testing was conducted using the RAGAS framework to assess the quality of answers and load testing for system performance. The results showed that the system achieved a Context Precision score of 0.86 and a Faithfulness score of 0.77, indicating high relevance and accuracy of answers. In addition, the implementation of the Groq API resulted in an average response latency of 2.11 seconds with a 94% success rate in load testing. These findings indicate that the RAG-based chatbot is effective in overcoming website limitations in delivering academic information at the Informatics Study Program at Muhammadiyah University of Sidoarjo.