AutoCT Analyst: AI-Augmented Scratch Assessment for Computational Thinking Development - GOH, KOK MING; SIRISENA, ANUTHRA (SEAMEO SEAMOLEC, 2025)
In the evolving landscape of primary computing education, accurately assessing students’ computational thinking (CT) remains a pedagogical challenge due to the dynamic nature of student learning and the creative expression embedded in Scratch programming. This study introduces AutoCT Analyst, an AI-powered tool designed to automate and personalize CT assessment by analyzing Scratch projects based on Brennan and Resnick’s CT framework. A total of 60 student-created Scratch artifacts were analyzed using AutoCT Analyst, which combines rule-based heuristics, decision-tree classification, and GPT-enhanced feedback. The tool identified four learner typologies—Novice Explorers, Linear Designers, Modular Thinkers, and Algorithmic Architects—each with specific scaffolding needs. Validation through expert raters demonstrated high reliability (Cohen’s κ = 0.82) and a strong correlation (r = 0.86) between human and AI-generated scores. Findings underscore the tool’s potential for large-scale, formative assessment and its capacity to inform differentiated instruction. The study also addresses the limitations of rigid typologies and highlights the necessity of human-AI hybrid feedback loops to balance creativity and structure in CT assessment. The commercial viability of AutoCT Analyst lies in its adaptability across classroom contexts and its ability to scale teacher feedback efficiently. Future research should explore longitudinal impacts and contextual customization across diverse educational ecosystems.