cover
Article Per Year (5 Year)
All
Home Page
OAI Link
Editorial Team
Contact
Reviewer
Google Scholar
Contact Name
-
Contact Email
-
Phone
-
Journal Mail Official
-
Editorial Address
-
Location
Kota malang,
Jawa timur
INDONESIA
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control
Published by Universitas Muhammadiyah Malang
ISSN : 25032259     EISSN : 25032267     DOI : -
Core Subject : Science, Engineering,
Computer Science & IT Control & Systems Engineering Electrical & Electronics Engineering Energy Engineering
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control was published by Universitas Muhammadiyah Malang. journal is open access journal in the field of Informatics and Electrical Engineering. This journal is available for researchers who want to improve their knowledge in those particular areas and intended to spread the knowledge as the result of studies. KINETIK journal is a scientific research journal for Informatics and Electrical Engineering. It is open for anyone who desire to develop knowledge based on qualified research in any field. Submitted papers are evaluated by anonymous referees by double-blind peer review for contribution, originality, relevance, and presentation. The Editor shall inform you of the results of the review as soon as possible, hopefully within 4 - 8 weeks. The research article submitted to this online journal will be peer-reviewed at least 2 (two) reviewers. The accepted research articles will be available online following the journal peer-reviewing process.
Arjuna Subject : -
Articles 581 Documents
 55   56   57   58   59 
Comparative Evaluation of BM25–FAISS and Small-LLM–GPT in Retrieval-Augmented Generation Concept Map Assessment Maskur Maskur; Didik Dwi Prasetya; Triyanna Widiyaningtyas; Azlan Mohd Zain
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control Vol. 11, No. 1, February 2026
Publisher : Universitas Muhammadiyah Malang

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.22219/kinetik.v11i1.2594

Abstract

Concept map-based assessment is a practical approach to measure students’ conceptual understanding, but manual assessment still faces challenges such as subjectivity, inconsistency, and limited scalability. This study proposes the application of Retrieval-Augmented Generation (RAG) as an artificial intelligence-based automated assessment solution in an educational context. The objectives of this study are to compare the effectiveness of two retrieval methods, BM25 and FAISS, and to analyse the trade-off between large-scale generative models (GPT) and Small-LLM in assessing concept map propositions. This study uses a quantitative experimental approach by combining a retriever and a generator in the RAG system. Performance evaluation is carried out using the Macro-F1 and QWK metrics to measure agreement with expert judgment, and the Explanation Relevance Score (ERS) to assess explanation quality. The experimental results show that the FAISS–GPT combination achieves the best performance, with a Macro-F1 of 0.338 and a QWK of 0.146, slightly superior to the BM25–GPT combination. In contrast, the use of Small-LLM, both with BM25 and FAISS, showed lower performance with Macro-F1 values in the range of 0.167–0.221 and QWK close to zero. This finding confirms that semantic-based retrieval plays a vital role in improving the accuracy of automated assessment, while large-scale generative models are more effective in representing conceptual relationships in depth. This study contributes through a comparative analysis of retrievers and generators, and by introducing ERS as an additional metric for RAG-based automated assessment in the field of education.

Page 59 of 59 | Total Record : 581

 55   56   57   58   59 

Filter by Year

2016 2026


Reset
Filter By Issues
All Issue Vol. 11, No. 3, August 2026 (Article in Progress) Vol. 11, No. 2, May 2026 Vol. 11, No. 1, February 2026 Vol. 10, No. 4, November 2025 Vol. 10, No. 3, August 2025 Vol. 10, No. 2, May 2025 Vol. 10, No. 1, February 2025 Vol. 9, No. 4, November 2024 Vol. 9, No. 3, August 2024 Vol. 9, No. 2, May 2024 Vol. 9, No. 1, February 2024 Vol. 8, No. 4, November 2023 Vol. 8, No. 3, August 2023 Vol. 8, No. 2, May 2023 Vol. 8, No. 1, February 2023 Vol. 7, No. 4, November 2022 Vol. 7, No. 3, August 2022 Vol. 7, No. 2, May 2022 Vol. 7, No. 1, February 2022 Vol. 6, No. 4, November 2021 Vo. 6, No. 3, August 2021 Vol. 6, No. 2, May 2021 Vol. 6, No. 1, February 2021 Vol. 5, No. 4, November 2020 Vol. 5, No. 3, August 2020 Vol. 5, No. 2, May 2020 Vol. 5, No. 1, February 2020 Vol 4, No 4, November 2019 Vol 4, No 3, August 2019 Vol 4, No 2, May 2019 Vol 4, No 2, May 2019 Vol 4, No 1, February 2019 Vol 4, No 1, February 2019 Vol 3, No 4, November 2018 Vol 3, No 4, November 2018 Vol 3, No 3, August 2018 Vol 3, No 3, August 2018 Vol 3, No 2, May-2018 Vol 3, No 2, May-2018 Vol 3, No 1, February-2018 Vol 3, No 1, February-2018 Vol 2, No 4, November-2017 Vol 2, No 4, November-2017 Vol 2, No 3, August-2017 Vol 2, No 3, August-2017 Vol 2, No 2, May-2017 Vol 2, No 2, May-2017 Vol 2, No 1, February-2017 Vol 2, No 1, February-2017 Vol 1, No 3, November-2016 Vol 1, No 3, November-2016 Vol 1, No 2, August-2016 Vol 1, No 2, August-2016 Vol 1, No 1, May-2016 Vol 1, No 1, May-2016 More Issue