Journal of Deep Learning, Computer Vision and Digital Image Processing
Volume 4 Issue 3 September 2026

Evaluating Precision and Semantic Similarity in a Retrieval-Augmented Generation-Based Question Answering System for Shafi'i Prayer Fiqh

Thoriq Wijaya (Universitas Malikussaleh)
Defry Hamdhana (Universitas Malikussaleh)
Fajriana (Universitas Malikussaleh)
Mukti Qamal (Universitas Malikussaleh)



Article Info

Publish Date
10 Sep 2026

Abstract

Purpose – This study evaluates retrieval precision and answer similarity in a closed-domain question-answering system for Shafi'i prayer fiqh using retrievalaugmented generation (RAG).Methods – Indonesian translations of three classical fiqh texts were segmented into 251 chunks using a 1000-character chunk size and 200-character overlap. LangChain coordinated multilingual-e5-base embeddings, ChromaDB retrieval, and local Qwen2.5:3b generation. The retriever returned up to four eligible chunks. Twenty predefined queries were evaluated using Precision@4, cosine similarity to researcher-prepared reference answers, and qualitative assessment of required answer components.Findings – Mean Precision@4 was 70.00%, and mean cosine similarity was 0.9197 (91.97%). Retrieval varied across queries, with Query 19 returning only one relevant chunk. Queries 8 and 19 produced incomplete answers despite high semantic similarity. The system returned a predefined fallback response when no retrieved passages passed the distance threshold.Research Implications – The results support preliminary prototype feasibility but do not establish expert-verified correctness of the rulings. The small query set and absence of a quantitative non-RAG baseline limit generalization and prevent estimation of any reduction in hallucination.Originality – The study evaluates retrieval and generation separately within a constrained Indonesian fiqh corpus and illustrates the distinction between semantic similarity and answer completeness.

Copyrights © 2026






Journal Info

Abbrev

DECODING

Publisher

Subject

Computer Science & IT

Description

The Journal of Deep Learning, Computer Vision and Digital Image Processing (DECODING), covers all topics of artificial intelligence and soft computing and their applications, including but not limited to: • Neural networks • Reasoning and evolution • Intelligent search • Intelligent planning ...