This research aims to enhance a Question-Answering System for Hadith texts by incorporating Refined Query techniques and Large Language Models (LLMs), specifically OpenAI's GPT-4. Utilizing a dataset of 62,169 Hadith from nine significant books, the study follows a comprehensive methodology that covers data collection, analysis and preprocessing, and the integration of LangChain and OpenAI's Chat Model for optimized querying. The evaluation of the system's performance was conducted through comparative analysis before and after the application of Refined Query, BERTScore for text quality, and user-based quality assessments. Results demonstrate that Refined Query significantly improves the system's capacity to produce accurate and contextually relevant responses. Implementing Refined Query not only enhanced answer precision but also facilitated the generation of responses where none were previously available. The average BERTScore of 0.80351 and the quality of user responses with an average score of 87.3% for the student test and 90.3% for the hadith expert test further validate the efficacy of the system. This research advances the domain of Islamic information systems by demonstrating the fruitful integration of advanced computational techniques with religious texts, offering a fundamental step towards better access to the understanding of Islamic jurisprudence.