Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)
Vol 10 No 5 (2026): October 2026 (In progress)

Benchmarking Transformer Models for Sentiment Analysis on YouTube User Comments

Li-Hua Li (Chaoyang University of Technology)
Agus Cahyo Nugroho (Universitas Katolik Soegijapranata)



Article Info

Publish Date
02 Oct 2026

Abstract

Social media has become one of the most important platforms for public expression due to its ability to provide users with an unprecedented scale for sharing opinions, sentiments, and experiences. Consequently, social media platforms provide valuable insights into public perception and user sentiment across various topics. This study presents a comparative evaluation of five transformer-based models, namely DistilBERT, BERT, BERTweet, RoBERTa, and Twitter-RoBERTa, on a cleaned and preprocessed YouTube Comments Dataset for sentiment analysis tasks. All models were evaluated under identical experimental settings and preprocessing procedures to reduce textual noise and improve data quality. Each transformer model was fine-tuned under common training conditions to assess its effectiveness in handling informal and user-generated social media content. Experimental results show that Twitter-RoBERTa outperforms other approaches and achieves the best overall accuracy: 90.13% (F1-score: 90.01%), demonstrating its ability to learn from contextual nuances and social media-specific language patterns. In addition, McNemar statistical analysis further supports the statistical significance of many performance differences between the models evaluated including those between Twitter-RoBERTa and smaller general-purpose models such as DistilBERT and BERT. These results demonstrate the value of using domain-specific pretraining on noisy social media text to improve sentiment classification performance and that transformer architectures built with fine-tuning in mind can effectively capture platform specific aspects of linguistics. In summary, this work presents a strong benchmark for transformer-based-sentiment-analysis on YouTube comments and valuable information for future research in social media sentiment analysis.

Copyrights © 2026






Journal Info

Abbrev

RESTI

Publisher

Subject

Computer Science & IT Engineering

Description

Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) dimaksudkan sebagai media kajian ilmiah hasil penelitian, pemikiran dan kajian analisis-kritis mengenai penelitian Rekayasa Sistem, Teknik Informatika/Teknologi Informasi, Manajemen Informatika dan Sistem Informasi. Sebagai bagian dari semangat ...