Article Per Year (5 Year)
p-Index From 2021 - 2026
1.713
P-Index
This Author published in this journals
All Journal International Journal of Advances in Applied Sciences Seminar Nasional Aplikasi Teknologi Informasi (SNATI) TELKOMNIKA (Telecommunication Computing Electronics and Control) Jurnal technoscientia Journal of ICT Research and Applications Jurnal Edukasi dan Penelitian Informatika (JEPIN) JOIV : International Journal on Informatics Visualization JURTEKSI Jurnal Ilmiah Realtech JURNAL TEKNOLOGI TECHNOSCIENTIA Jurnal Sains Indonesia Jurnal Pengabdian Masyarakat Nusantara IT For Society : Journal of Information Technology Eduvest - Journal of Universal Studies
Rila Mandala
School Of Electrical Engineering And Informatics, Bandung Institute Of Technology
Aerospace Engineering Humanities Chemical Engineering, Chemistry & Bioengineering Civil Engineering, Building, Construction & Architecture Computer Science & IT Control & Systems Engineering Earth & Planetary Sciences Education Energy Engineering Environmental Science Health Professions Industrial & Manufacturing Engineering Materials Science & Nanotechnology Mathematics Mechanical Engineering Neuroscience Physics Social Sciences Other

Published : 31 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 31 Documents
Search

 1   2   3   4 
OPTIMIZING RETRIEVAL-AUGMENTED GENERATION FOR DOMAIN-SPECIFIC KNOWLEDGE SYSTEMS THROUGH FINE-TUNING AND PROMPT ENGINEERING Ahmad Fajri; Rila Mandala
JURTEKSI (jurnal Teknologi dan Sistem Informasi) Vol. 12 No. 1 (2025): Desember 2025
Publisher : Lembaga Penelitian dan Pengabdian Kepada Masyarakat (LPPM) STMIK Royal Kisaran

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.33330/jurteksi.v12i1.4338

Abstract

Abstract: This study discusses the optimization of RAG for a FAQ system in the field of information technology product security certification at BSSN. Although LLM generate reliable responses, they often lack up-to-date and domain-specific knowledge, which can be addressed through the RAG approach. This research aims to optimize a domain-specific RAG system by improving embedding performance, enhancing prompt robustness, and increasing retrieval accuracy. The research methods consist of three stages. The first stage involves fine-tuning the bge-m3 embedding model and evaluating its performance using MRR, Recall, and AUC. The second stage applies prompt engineering techniques, namely the SRSM and Autodefense, to mitigate direct-injection and escape-character prompt injection attacks. The third stage evaluates the proposed RAG system using Precision, Recall, and F1-Score metrics against four baseline models. The results of research show that the fine-tuned embedding model achieves higher performance than the original model, with MRR@1 and Recall@1 values of 0.80 and an AUC@100 of 0.7023. In addition, the proposed prompt engineering techniques demonstrate robustness against prompt injection attacks, while the overall RAG system attains a perfect Precision, Recall, and F1-Score of 1.00. In conclusion, the proposed approach effectively enhances retrieval accuracy, embedding quality, and system security, resulting in a more reliable RAG-based FAQ system for information technology product security certification. Keywords: embedding fine-tuning; large language model; prompt engineering; prompt injection mitigation; retrieval-augmented generation Abstrak: Studi ini membahas optimasi RAG untuk sistem FAQ di bidang sertifikasi keamanan produk teknologi informasi di BSSN. Meskipun LLM menghasilkan respons yang andal, mereka seringkali kurang memiliki pengetahuan terkini dan spesifik domain, yang dapat diatasi melalui pendekatan RAG. Penelitian ini bertujuan untuk mengoptimalkan sistem RAG spesifik domain dengan meningkatkan kinerja embedding, meningkatkan ketahanan prompt dan meningkatkan akurasi pengambilan. Metode penelitian terdiri dari tiga tahap. Tahap pertama melibatkan fine-tuning model embedding bge-m3 dan mengevaluasi kinerjanya menggunakan Mean Reciprocal Rank (MRR), Recall, dan AUC. Tahap kedua menerapkan teknik rekayasa prompt, yaitu Self- SRSM dan Autodefense, untuk mengurangi serangan direct-injection dan escape-character prompt injection. Tahap ketiga mengevaluasi sistem RAG yang diusulkan menggunakan metrik Presisi, Recall, dan F1-Score terhadap empat model dasar. Hasil penelitian menunjukkan bahwa model embedding yang disempurnakan mencapai kinerja yang lebih tinggi daripada model asli, dengan nilai MRR@1 dan Recall@1 sebesar 0,80 dan AUC@100 sebesar 0,7023. Selain itu, teknik rekayasa prompt yang diusulkan menunjukkan ketahanan terhadap serangan injeksi prompt, sementara sistem RAG secara keseluruhan mencapai Presisi, Recall, dan F1-Score sempurna sebesar 1,00. Kesimpulannya, pendekatan yang diusulkan secara efektif meningkatkan akurasi pengambilan, kualitas embedding dan keamanan sistem, menghasilkan sistem FAQ berbasis RAG yang lebih andal untuk sertifikasi keamanan produk teknologi informasi. Kata kunci: penyempurnaan embedding; model bahasa besar; rekayasa prompt; mitigasi injeksi prompt; retrieval-augmented generation
Co-Authors Abdurrahman - Agus Priyanto Agus Susanto Ahmad Fajri Akbar, Saiful Alfonso Mario Tumewu Andreas Prasetia Antonius Sigit Arfan As’Sidiq Arry Akhmad Arman Audy Aldrin Kenap Ayi Purbasari Bambang Riyanto Trilaksono Bambang Riyanto Trilaksono Chong Min An Debby Paseru Dejan Kalengkongan Deny Haryadi Dewi Marini Umi Atmaja Dwi H. Widyantoro Erry Koryanti Fahmi, Hasanul Handwi, Narendra Rahman Harlili Harlili Iping Supriana Suwardi Irfan Nasrullah Ivana Masala Kalengkongan, Dejan Masala, Ivana Michael Aleng Min An, Chong Mohammad Deny Safari Muhammad Erwin Ashari Hariyono Muhammad Ihsan Oerip Slamet Santoso Rajesri Govindaraju Rasyid, Abdul Aziz Al Riady, Sasmitoh Rahmad Rinaldi Munir Rosalina Rosalina Sahuri, Genta Sahuri Sengkey, Axel Shita Dwi Ranti Tjong Wan Sen Zainal Abidin