<![CDATA[Large Language Models (LLMs) can generate fluent responses but may produce unsupported answers in domain-specific settings. This study developed and evaluated a question answering application for electronic-store information retrieval in Palembang by integrating Qwen3-8B with Retrieval-Augmented Generation (RAG). Using Design Science Research, the study produced a web application supported by a knowledge base of 1,019 product records from eight stores. Each record contained six attributes: store name, address, telephone number, Google Maps link, category, and product name. The data were converted into 768-dimensional embeddings using sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 and stored in ChromaDB. The system retrieved the top five records based on cosine similarity and applied a 0.65 threshold before supplying context to Qwen3-8B through the Groq API. Evaluation used 200 queries; positive labels indicated available information, whereas negative labels indicated unavailable or out-of-scope information. Responses were classified as true positives (TP) when positive queries were answered correctly, true negatives (TN) when negative queries were rejected correctly, false positives (FP) when negative queries received answers, and false negatives (FN) when positive queries were rejected or not answered. Testing yielded TP=143, TN=5, FP=34, and FN=18, with 74.00% accuracy, 80.79% precision, 88.82% recall, and an 84.61% F1-score. Errors occurred with short ambiguous queries, semantic overlap, local terminology, and typographical variation. RAG supplied retrieved context as the basis for responses, but rejection of queries outside the knowledge base remains limited.]]>
Copyrights © 2026
