Detecting AI-Generated Text with Fine-Tuned RoBERTa: A Cross-Validated Study with Interpretability Analysis - Yuhefizar Yuhefizar; Ronal Watrianthos; Dony Marzuki (Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), 2026)
The rapid proliferation of large language models capable of generating fluent and contextually coherent text has made reliable authorship attribution both a practical and ethical necessity. Outputs from systems such as GPT-4, LLaMA 2, and Claude are increasingly indistinguishable from human-authored prose, yet current detection methodologies remain insufficient to meet this challenge. This study aims to develop and evaluate a fine-tuned RoBERTa-base binary classifier for distinguishing AI-generated text from human-authored text under stratified cross-validation conditions; assess the model's robustness to a known length-based confound through a controlled ablation study; and provide interpretable explanations of the classifier's decisions through dual token-level attribution using Gradient × Input Saliency and Attention Rollout. The model is trained and evaluated via stratified five-fold cross-validation on a balanced corpus of 6,069 samples (3,069 AI-generated and 3,000 human-authored) and benchmarked against four established baselines spanning classical, neural, and zero-shot paradigms. The classifier achieves a mean accuracy of 99.93%, a weighted F1-score of 0.9993, and a mean AUC-ROC of 1.0000 across all five folds. A length-balanced ablation produces an accuracy decline of only 0.11 percentage points, providing empirical evidence that the model has learned genuine semantic and stylistic distinctions rather than superficial length-based cues. Gradient × Input Saliency maps indicate that AI-generated texts are characterised by technically precise, domain-specific vocabulary, whereas human-authored texts are identified through discourse markers and rhetorically structured expression, with both attribution methods converging on the same high-saliency tokens.