<![CDATA[This study evaluates the reliability of confidence estimates produced by a Vanilla BERT classifier for stress detection using the Dreaddit benchmark. BERT-base-uncased was fine-tuned on 3,553 labeled text segments, following the standard split of 2,838 training samples and 715 test samples. The model was assessed as a single diagnostic baseline without additional linguistic features, label smoothing, post-hoc calibration, or other calibration interventions. Evaluation was conducted using discriminative performance metrics, including accuracy, precision, recall, and F1-score, as well as probabilistic reliability metrics, including Brier Score, Expected Calibration Error, Adaptive Calibration Error, and a reliability diagram. The Vanilla BERT model achieved 79.02% accuracy, 78.00% precision, 82.65% recall, and 80.26% F1-score, indicating competitive classification performance for stress detection. However, the calibration results revealed noticeable miscalibration, with a Brier Score of 0.1565, Expected Calibration Error of 0.0847, and Adaptive Calibration Error of 0.0880. The most prominent confidence mismatch occurred in the 0.8–0.9 confidence interval, while the 0.9–1.0 interval contributed the most to Expected Calibration Error due to its larger sample proportion. These findings show that although Vanilla BERT performs reasonably well in distinguishing stressed from non-stressed text, its confidence estimates are not fully reliable. Therefore, this study positions Vanilla BERT as a diagnostic reliability baseline and emphasizes the importance of evaluating stress detection models using both classification performance and probabilistic calibration criteria.]]>
Copyrights © 2026
