Sinkron : Jurnal dan Penelitian Teknik Informatika
Vol. 10 No. 4 (2026): Research Articles October 2026

Cost-Aware Selective Text Classification with Calibrated Lightweight Models

Saif Alhusseny (Department of Physics, College of Education for Pure Science, University of Mosul)
Omar Alniemi (Department of Computer Science, College of Education for Pure Science, University of Mosul)
Alaa Saadoon (Department of Computer Science, College of Education for Pure Science, University of Mosul)
Hanaa Mahmood (Department of Computer Science, College of Education for Pure Science, University of Mosul)



Article Info

Publish Date
05 Oct 2026

Abstract

Reliable text classification requires calibrated confidence, abstention for uncertain cases, and explicit computational-cost reporting. This study evaluates cost-aware selective prediction for calibrated lightweight text classifiers on commodity CPU runtimes. The goal is to find a light-weight classifier that is both deployment friendly and achieves a balance between predictive accuracy, calibration, selective reliability, and computational efficiency. The pipeline trains Multinomial Naive Bayes, Logistic Regression, and LightGBM on TF-IDF features, applies post-hoc probability calibration, and accepts only predictions above a validation-selected confidence threshold. Calibrated Logistic Regression achieves the best practical accuracy-calibration-selective-risk-latency-review-budget trade-off in SMS spam, IMDb sentiment and AGNews topic classification experiments. Calibrated Logistic Regression achieves 0.9857 accuracy and 0.0070 ECE on SMS, 0.8718 accuracy and 0.0162 ECE on IMDb, and 0.8998 accuracy on AGNews. On IMDb, Logistic Regression sacrifices about one accuracy point relative to LightGBM but reduces p95 CPU latency from 2546.06 ms to 3.26 ms. Abstaining on the lowest confidence predictions moves accepted risk down from 0.0143 to 0.00457 (SMS) and 0.1282 to 0.1006 (IMDb). Thresholds are chosen on validation/calibration data, then frozen for a single test run. More generally, the results establish calibrated lightweight classifiers as strong and transparent baselines for cost-effective low-latency text classification in real-world deployments.

Copyrights © 2026






Journal Info

Abbrev

sinkron

Publisher

Subject

Computer Science & IT

Description

Scope of SinkrOns Scientific Discussion 1. Machine Learning 2. Cryptography 3. Steganography 4. Digital Image Processing 5. Networking 6. Security 7. Algorithm and Programming 8. Computer Vision 9. Troubleshooting 10. Internet and E-Commerce 11. Artificial Intelligence 12. Data Mining 13. Artificial ...