Aprizawati Aprizawati
Department Maritime, Polytechnic State of Bengkalis, Riau, Indonesia

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

Development of Smart Assessment System For Evaluating Maritime English Competence Using Machine Learning Aprizawati Aprizawati; Romadhoni Romadhoni; Budhisantoso Budhisantoso
AL-ISHLAH: Jurnal Pendidikan Vol 18, No 1 (2026): MARCH 2026
Publisher : STAI Hubbulwathan Duri

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.35445/alishlah.v18i1.9624

Abstract

Maritime English proficiency assessment is essential for cadets and maritime professionals, yet manual scoring can be time-consuming and prone to inter-rater variability. This study proposes a web-based smart assessment system that integrates machine learning to classify Maritime English proficiency into Beginner/Intermediate/Advanced using four feature scores: listening, reading, writing, and speaking. The dataset used in this work is simulated (1,000 records) for proof-of-concept evaluation because access to large, standardized real examination data was limited and required institutional clearance; simulation enables controlled class balance and repeatable experimentation. Class labels are generated using rubric-based threshold rules, and the labeling scheme is validated by two Maritime English examiners who review the thresholds and independently rate a random subset of 200 samples; agreement is quantified using Cohen’s kappa (κ) to ensure reliability. We adopt an 80/20 hold-out split and apply stratified 5-fold cross-validation on the training set for model selection, using grid-search hyperparameter tuning. We compare Support Vector Machine (SVM) and Random Forest and report accuracy, precision, recall, macro-F1, and brief per-class performance for Beginner/Intermediate/Advanced. SVM achieves 92% accuracy with macro-F1 = 0.905, outperforming Random Forest (89%, macro-F1 = 0.875). Future work will validate the system using real assessment datasets in operational training settings.