<![CDATA[Purpose - This study evaluates an auditable hybrid machine-learning workflow for sentiment classification and operational hate-speech detection in Indonesian political YouTube comments, distinguishing agreement with lexicon-derived labels from agreement with human judgments. Methods - From 25,651 comments, 25,484 were retained. The sentiment target was reconstructed from a 10,272-row polarity lexicon, and the operational target was audited against a 545-term lexicon. Unigram-bigram TF-IDF and lexical features were tested using Multinomial Naive Bayes, linear SVM, Random Forest, and hard voting. Evaluation used an 80:20 holdout, five-fold validation, ablation, keyword masking, McNemar tests, a blinded 400-comment dual-annotation reference set with adjudication, and out-of-fold regional prediction. Findings - Linear SVM achieved weighted F1 scores of 83.29% for sentiment and 99.03% for the operational hate-speech target. On the human reference set, SVM without hate-speech features achieved 66.50% accuracy and 18.29% hate-class F1; the operational rule achieved 18.93% hate-class F1. Annotator agreement was 82.25% with Cohen's kappa of 0.1834. Neutral sentiment remained the weakest class. Regional estimates differed by at most 1.93 percentage points between SVM and voting. Research implications - Near-ceiling operational-target performance does not establish general moderation accuracy. The workflow supports exploratory screening and human-review prioritization. Regional outputs describe source-video topic groups, not provincial populations. Originality - The study integrates target reconstruction, leakage-aware testing, human-reference evaluation, model comparison, and out-of-fold regional visualization in one reproducible workflow.]]>
Copyrights © 2026
