Analytical Framework for Evaluating AI Detection Performance Across Writing Modalities in Technical English Writing

Agustina Ginting (Politeknik Negeri Medan)
Harris Aminuddin (Politeknik Negeri Medan)
Ulfa Hasnita (Politeknik Negeri Medan)

Information Technology Education Journal
Vol. 5, No. 2, May (2026)

Purpose – This study proposes a conceptual framework for AI-based detection of handwritten and technical English writing in engineering education by investigating how different writing modalities influence AI detection performance. Design/methodology/approach – The proposed framework integrates four representative writing modalities, namely handwritten, typed, hybrid, and transcribed texts. A framework-based analytical evaluation was conducted using representative detection scores together with accuracy, precision, recall, F1-score, confidence intervals, and heatmap visualization to compare AI detection performance across writing modalities. Findings – The analytical evaluation suggests that AI detection performance varies across writing modalities, with representative accuracy ranging from 0.72 to 0.89. Detection scores progressively increase from handwritten (0.41–0.50) to typed (0.52–0.58), hybrid (0.60–0.66), and transcribed texts (0.68–0.76). Transcribed texts demonstrate the highest performance, with precision and recall reaching 0.87–0.89, whereas handwritten texts exhibit lower recall (0.44) and broader confidence intervals, indicating greater classification uncertainty. Hybrid writing shows intermediate performance due to overlapping human and AI-assisted linguistic characteristics. Research implications/limitations – This study is limited to a framework-based analytical evaluation using representative writing scenarios rather than empirical observations. Nevertheless, the proposed framework provides practical guidance for developing context-aware AI detection systems that improve fairness and reliability in evaluating technical and English as a Second Language (ESL) writing. Originality/value – This research presents an integrated analytical framework that combines multiple writing modalities with AI detection performance evaluation to investigate modality-dependent differences in technical English writing. The proposed framework contributes to the development of more robust, context-aware, and equitable AI-assisted writing assessment for engineering education.