Evaluasi dan Implementasi Indobert Question Answering (QA) pada Domain Spesifik Menggunakan Mean Reciprocal Rank

Ramadhan, Teguh Ikhlas
Supriatman, Agus
Kurniawan, Taufik Rahmat

Jurnal Algoritma
Vol 21 No 1 (2024): Jurnal Algoritma

Applications of artificial intelligence, such as Chat-GPT and Bard, have become common in various aspects of life today. One of the main aspects is the use of the Question Answering (QA) model to meet specific domain needs. However, in some cases, models like Bard may not be able to provide specific information, such as registration procedures at a particular company or lecturer schedules at a university. To overcome this challenge, an Indonesian QA model called IndoBERT-QA has been developed. This research aims to evaluate the capabilities of IndoBERT-QA in a specific domain context using the Mean Reciprocal Rank (MRR) evaluation method. The evaluation results show that the IndoBERT-QA model is able to achieve an MRR of 0.91 with an approach to creating a customized context for each question. These results indicate that this model has good performance in providing relevant answers. The benefit of this research is that it serves as a valuable reference for the development of QA systems that will be created by other parties in the same environment. By utilizing a well-tested and evaluated approach, this research provides a strong foundation for the development of a high-performance QA system in Indonesian, so that it can meet domain-specific needs. Besides that, this research also provides insights for further development. One approach that can be explored is the use of Information Retrieval or Passage Retrieval as an initial step in the QA process. This can help the model in getting more precise and relevant context, thereby allowing further improvement in the quality of the answers provided by the model. Index Terms-BERT, Mean Reciprocal Rank, Question Answering However, this research also provides insights for further development. One approach that can be explored is the use of Information Retrieval or Passage Retrieval as an initial step in the QA process. This can help the model in getting more precise and relevant context, thereby allowing further improvement in the quality of the answers provided by the model.