Evaluating Test Quality in English for Mathematics Course: An Item Analysis Approach - Hedia Rizki; Reri Seprina Anggraini; Indra Bagus Satrio (Universitas Islam Sultan Agung, 2026)
This study evaluated the quality of an English for Mathematics course administered to 35 third-semester students in the Mathematics Education Study Program at UIN Sulthan Thaha Saifuddin Jambi using Classical Test Theory. The 50-item multiple-choice test covered eight mathematical topics, including numbers, rational numbers, powers and roots, statistics and probability, logic, algebra, geometry, and Measurement. Item analysis examined difficulty indices, discrimination indices, reliability (KR-20), and distractor effectiveness. Results revealed excellent reliability (r₁₁ = 0.911) and balanced difficulty distribution: 36% easy items (P > 0.70), 52% moderate items (0.30-0.70), and 12% difficult items (P < 0.30). The majority of items (86%) demonstrated acceptable discrimination capacity, with 56% achieving excellent discrimination (D ≥ 0.40). However, critical concerns emerged: difficult items clustered in geometry and measurement topics (items 39-44, 48), 14% of items showed poor discrimination, and 27% of distractors proved ineffective. Findings indicate that the instrument possesses strong overall technical quality, while requiring targeted revisions to poorly discriminating items and ineffective distractors. The study recommends enhanced instructional support for geometry and measurement topics, qualitative analysis of student reasoning, and future research employing differential item functioning analysis to distinguish mathematical ability from language proficiency effects in English for Mathematics assessment. Keywords: Classical Test Theory, English for Mathematic, Evaluation