Enhancing AI-driven Cybersecurity Awareness Smart Consultant using RAG Method with Hybrid Knowledge Based - Pratama, I Putu Agus Eka; Widyantara, I Made Oka; Linawati, Linawati; Gunantara, Nyoman (International Journal of Advances in Data and Information Systems, 2026)
The rapid advancement of Artificial Intelligence (AI), particularly in the realm of Large Language Model (LLM), holds significant potential for addressing the escalating issue of cyberattacks that exploit users' insufficient cybersecurity awareness. This study involves the design and development of a prototype cybersecurity awareness smart consultant, leveraging AI through the Retrieval Augmented Generation (RAG) method. This approach integrates hybrid knowledge derived from both user-specific internal cybersecurity documents and internet resources, thereby enhancing the validity of system responses and mitigating the risk of hallucinations. The prototype was evaluated using the Answer Accuracy Score (AAS) method, based on Black Box Testing and human evaluation, across four cybersecurity-related questions, yielding an average score of 0.925, accompanied by comprehensive analysis and discussion. The findings indicate that the system's response accuracy improves when knowledge is synthesized from both internal document resources and internet sources. Future research may focus on incorporating deliberative thinking to further enhance system performance in generating responses.