Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal Bulletin of Network Engineer and Informatics (BUFNETS)
Hairatunnisa
Universitas Ahmad Dahlan
Computer Science & IT

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Comparative Evaluation of Reasoning Consistency Detection Methods on LLM Outputs in Cardiovascular Clinical Scenarios Hairatunnisa; Herman; Muhammad Kunta Biddinika
Bulletin of Network Engineer and Informatics Vol. 4 No. 2 (2026): BUFNETS (Bulletin of Network Engineer and Informatics) October 2026
Publisher : PT. GWEX NET PUBLISHER

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.59688/w2r4vk72

Abstract

The advancement of Large Language Models (LLMs) presents opportunities for developing cognitive assistants in the medical field, particularly in cardiovascular care.However, hallucinations and reasoning inconsistencies remain significant challenges, while the evaluation of logical consistency in this domain is limited due to the absence of public datasets with clear ground truth. This study aims to compare the effectiveness of Self-Consistency (LLM), Sentiment Heuristic, and Keyword Overlap methods in detecting reasoning inconsistencies using a synthetic dataset of 90 cardiovascular clinical scenarios (30 scenarios with 3 variants each). Performance was evaluated based on accuracy, precision, recall, and F1-score, and benchmarked against a fine-tuning approach. The results show that the zero-shot detector based on self-consistency achieved the best performance, with an accuracy of 97.78% and an F1-score of 98.31%, while the fine-tuning approach on the limited synthetic dataset failed to surpass these results. These findings indicate that the semantic reasoning capabilities of LLMs have the potential to serve as consistency detectors without additional training, and can serve as a foundation for developing cognitive assistant systems in the cardiovascular domain.
Co-Authors Herman Muhammad Kunta Biddinika