<![CDATA[This study aims to analyze the consistency characteristics of logical fallacy identification by a Large Language Model (LLM) and to identify the types and distribution of logical fallacies in students' undergraduate theses. The research employed a qualitative descriptive design supported by quantitative data. The sample consisted of six theses written by 2021-cohort Indonesian Language and Literature Education students at Universitas Jambi, selected through purposive sampling with a maximum-variation principle across three fields of study (Teaching, Linguistics, and Literature). Each argumentative chapter (Chapters I, II, IV, V) was analyzed using Claude Opus 4.6 across three independent runs with identical prompts, yielding 72 analysis sets. Consistency was measured using TARa (Total Agreement Rate at parsed-answer level) within an intra-rater reliability framework, while the qualitative data were analyzed with the Miles & Huberman (1994) model. The findings show that LLM identification consistency is low overall yet patterned (pooled TARa 31.58%; 24 out of 76 consistent instances): Chapter II attained the highest TARa (70.00%) and Chapter IV the lowest (3.45%). Among the 24 consistent instances, four fallacy types were identified: False Cause (54.17%), Begging the Question (25.00%), Hasty Generalization (16.67%), and Missing the Point (4.17%); no fallacies of Ambiguity were detected. The Teaching field contained the most instances (54.17%), and Chapter II was the most vulnerable structural component (58.33%). These findings extend Harjono’s (2011) work on relevance fallacies and provide an empirical basis for targeted pedagogical interventions to improve students’ logical reasoning, particularly in distinguishing correlation from causation and avoiding circular reasoning.]]>
Copyrights © 2026
