Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal International Journal of Evaluation and Research in Education (IJERE)
JiEun Janet Song
Korea University
Education

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Image-native automated scoring of handwritten mathematical responses: reliability evidence and teacher–AI collaboration JiEun Janet Song; Young-seok Oh; Dong Joong Kim
International Journal of Evaluation and Research in Education (IJERE) Vol 15, No 4: August 2026
Publisher : Institute of Advanced Engineering and Science

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.11591/ijere.v15i4.38456

Abstract

This study examines the reliability of an image-native multimodal AI system for automated scoring of handwritten responses to Advanced Placement (AP) Calculus free-response items without requiring optical character recognition (OCR) preprocessing. Using inter-rater agreement indices and test–retest reliability analyses, we found substantial to almost perfect agreement between artificial intelligence (AI)-generated scores and calibrated human ratings, as well as almost perfect stability across repeated scoring sessions. These results suggest that the observed reliability of the AI scoring system warrants further investigation of validity-related evidence and inferences. As a practical implication for assessment practice, we propose a human-in-the-loop teacher–AI collaborative (TAC) framework in which automated scoring operates under teacher oversight. Taken together, these findings provide initial evidence of reliability supporting the responsible use of AI-based scoring as a measurement instrument in high-stakes educational assessment.
Co-Authors Dong Joong Kim Young-seok Oh