<![CDATA[The dropout of students is a significant problem in the higher education sector, affecting the graduation rate and causing serious academic and financial implications for students and institutions. In this sense, the early prediction of students that are more likely to drop out of school is necessary for the development of strategies to prevent such an outcome. Here we present a deep learning model based on a transformer architecture to predict the dropout of students using structured educational data. The proposed model uses the Feature Tokenization Transformer (FTTransformer) with self-attention mechanisms. The dataset used was the Predict Students Dropout and Academic Success. To train the model, a pre-processing pipeline was built that included the standardization of features, median imputation, and the creation of new features that helped to identify deviations in academic performance and financial risk. The performance of the model was evaluated on a test dataset, with the following metrics: accuracy (0.881), F1-score (0.88), ROC-AUC (0.9349) and PR-AUC (0.9044). In addition, permutation feature importance was used to identify the most important features related to the dropout prediction. The results show that the tuition payment status and academic performance had the highest importance in the model. The results presented demonstrate the performance of transformer-based models and the use of interpretable deep learning models to support the decision making of higher education institutions.]]>
Copyrights © 2027
