<![CDATA[This study examines public sentiment toward vocational higher education in Indonesia using 2,014 YouTube comments from 79 videos and evaluates Naïve Bayes, Support Vector Machine (SVM), and Random Forest classifiers against human-validated labels. A vocational higher-education lecturer and an Indonesian-language lecturer annotated all comments independently. They initially agreed on 1,958 comments (97.22%; Cohen’s κ = 0.945), and resolved 56 disagreements by consensus. The final labels comprised 1,227 neutral (60.92%), 702 positive (34.86%), and 85 negative comments (4.22%). A stratified 80/20 split was created before any balancing; the 403-comment test set retained the natural distribution. On the training partition, unigram versus unigram, bigram features and no balancing, class weighting, or training-only undersampling were compared using repeated stratified five-fold cross-validation (two repeats). SVM with unigram TF-IDF and balanced class weights achieved the best cross-validated macro-F1 among the three selected model configurations (0.650 ± 0.054). On the untouched test set, SVM obtained 76.67% accuracy and 0.655 macro-F1, and recovered 6 of 17 negative comments (35.29% recall). Random Forest produced slightly higher accuracy (77.17%) but only 5.88% negative recall, while Naïve Bayes detected no negative cases. The results provide a reproducible screening pipeline that may complement institutional accountability mechanisms, but the limited recovery of negative feedback precludes its use as an autonomous early-warning system.]]>
Copyrights © 2026
