Journal of Information Systems Engineering and Business Intelligence
Vol. 12 No. 2 (2026): June

A Dual-Stream CNN and Trajectory-Transformer Model for Early Dysgraphia Screening Using Handwritten Data

Yuri Pamungkas (Institut Teknologi Sepuluh Nopember, Surabaya)
Abdul Karim (Hallym University, Chuncheon)
Muhammad Nur Afnan Uda (Universiti Malaysia Sabah, Kinabalu)
Uda Hashim (Universiti Malaysia Sabah, Kinabalu)



Article Info

Publish Date
07 Jul 2026

Abstract

Background: Dysgraphia is a learning disorder that commonly affects handwriting fluency and legibility through difficulties in motor coordination and spatial organization. Early identification is needed to support timely intervention, but conventional assessment remains dependent on subjective observation and manual evaluation, making it difficult to apply efficiently at scale. To address this problem, this study proposes a dual-stream convolutional neural network and trajectory-transformer model for automated early dysgraphia screening by combining spatial and temporal handwriting features Objective: This study develops an end-to-end multimodal deep learning framework that integrates image- and trajectory-based handwriting representations to support accurate and interpretable dysgraphia classification. Methods: The proposed model contains two complementary streams. The CNN stream extracts spatial handwriting features, including stroke shape, character structure, and alignment, whereas the transformer stream models temporal movement patterns, such as stroke rhythm and writing velocity. These representations are combined through an attention-based fusion mechanism to produce a unified SEM. The model was trained and evaluated using the Potential Dysgraphia Handwriting Dataset, which includes 249 labeled handwriting samples categorized as low and potential dysgraphia. Results: The model achieved an overall accuracy of 95.9%, with precision, recall, and F1-score values of approximately 0.96 and an area under the curve (AUC) of 0.997. It also outperformed single-stream baseline models. Grad–CAM and attention map visualizations showed that the model focused on dysgraphia-associated handwriting regions and ischemic stroke patterns. The t–SNE projection of fused features showed clear separation between the two classes, indicating that the learned embeddings contained discriminative spatial and temporal information. Conclusion: The dual-stream convolutional neural network and trajectory-transformer model provides an accurate and explainable approach for early dysgraphia detection. The framework offers a data-driven basis for objective handwriting assessment in educational and clinical settings by linking the visual structure of handwriting with the movement process used to produce it.   Keywords: Dysgraphia screening, Handwriting analysis, Convolutional Neural Network (CNN), Transformer, Multimodal deep learning

Copyrights © 2026






Journal Info

Abbrev

JISEBI

Publisher

Subject

Computer Science & IT

Description

Jurnal ini menerima makalah ilmiah dengan fokus pada Rekayasa Sistem Informasi ( Information System Engineering) dan Sistem Bisnis Cerdas (Business Intelligence) Rekayasa Sistem Informasi ( Information System Engineering) adalah Pendekatan multidisiplin terhadap aktifitas yang berkaitan dengan ...