Thoriq Wijaya
Universitas Malikussaleh

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

Evaluating Precision and Semantic Similarity in a Retrieval-Augmented Generation-Based Question Answering System for Shafi'i Prayer Fiqh Thoriq Wijaya; Defry Hamdhana; Fajriana; Mukti Qamal
Journal of Deep Learning, Computer Vision, and Digital Image Processing Volume 4 Issue 3 September 2026
Publisher : CV. Sakura Digital Nusantara

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.61255/decoding.v4i3.1840

Abstract

Purpose – This study evaluates retrieval precision and answer similarity in a closed-domain question-answering system for Shafi'i prayer fiqh using retrievalaugmented generation (RAG).Methods – Indonesian translations of three classical fiqh texts were segmented into 251 chunks using a 1000-character chunk size and 200-character overlap. LangChain coordinated multilingual-e5-base embeddings, ChromaDB retrieval, and local Qwen2.5:3b generation. The retriever returned up to four eligible chunks. Twenty predefined queries were evaluated using Precision@4, cosine similarity to researcher-prepared reference answers, and qualitative assessment of required answer components.Findings – Mean Precision@4 was 70.00%, and mean cosine similarity was 0.9197 (91.97%). Retrieval varied across queries, with Query 19 returning only one relevant chunk. Queries 8 and 19 produced incomplete answers despite high semantic similarity. The system returned a predefined fallback response when no retrieved passages passed the distance threshold.Research Implications – The results support preliminary prototype feasibility but do not establish expert-verified correctness of the rulings. The small query set and absence of a quantitative non-RAG baseline limit generalization and prevent estimation of any reduction in hallucination.Originality – The study evaluates retrieval and generation separately within a constrained Indonesian fiqh corpus and illustrates the distinction between semantic similarity and answer completeness.