Mochammad Arif Bijaksana
School of Computing, Telkom University, Indonesia

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

Cost-Efficient Strategies for Production Indonesian Sentiment Analysis: Open-Source Models, Commercial LLMs, and Hybrid Pipelines Ridwan Ilyas; Mochammad Arif Bijaksana
Jurnal Teknik Informatika (Jutif) Vol. 7 No. 5 (2026): JUTIF Volume 7, Number 5, October 2026
Publisher : Informatika, Universitas Jenderal Soedirman

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.52436/1.jutif.2026.7.5.6297

Abstract

Production sentiment analysis for Indonesian must balance accuracy against operating cost, yet prior work benchmarks accuracy in isolation. This study aims to quantify how much accuracy is purchased per unit of monetary cost across three pipeline families, and to test whether hybrid pipelines can approach large language model accuracy at a fraction of the cost. We fine-tune four open-source encoders (IndoBERT-base, IndoBERT-large with Low-Rank Adaptation, IndoBERTweet, and IndoBERT-lite), evaluate four commercial large language model arms (Claude Haiku, Sonnet, and Opus under zero-shot and few-shot prompting), and build three hybrid pipelines—confidence routing, uncertainty cascading, and knowledge distillation—on four Indonesian sentiment datasets (SmSA, NusaX, UGM Twitter, and ID-SMSA). Accuracy is measured by Macro-F1 over three seeds, cost by a per-1,000-prediction model, and differences by McNemar's test with bootstrap confidence intervals. Paying for a large language model is statistically significant on only one of four datasets: on SmSA the best open-source encoder reaches 0.924 against 0.966 for Sonnet, whereas on the noisy UGM corpus the free encoder (0.708) significantly outperforms every paid model (≤0.651, p<0.001). Confidence routing is the consistent winner, reaching 0.947 on SmSA at only US$0.18 per 1,000 predictions—about 98% of full-model accuracy at one-ninth of the cost—and surpassing both pure endpoints on three of four datasets. The contribution is a statistically grounded, reproducible decision framework for selecting sentiment pipelines in low-resource languages that specifies when paying for large language models is justified and when free fine-tuned encoders suffice.