Jurnal Teknik Informatika (JUTIF)
Vol. 7 No. 5 (2026): JUTIF Volume 7, Number 5, October 2026

Cost-Efficient Strategies for Production Indonesian Sentiment Analysis: Open-Source Models, Commercial LLMs, and Hybrid Pipelines

Ridwan Ilyas (Informatics Department, Faculty of Science and Informatics, Universitas Jenderal Achmad Yani, Indonesia)
Mochammad Arif Bijaksana (School of Computing, Telkom University, Indonesia)



Article Info

Publish Date
10 Oct 2026

Abstract

Production sentiment analysis for Indonesian must balance accuracy against operating cost, yet prior work benchmarks accuracy in isolation. This study aims to quantify how much accuracy is purchased per unit of monetary cost across three pipeline families, and to test whether hybrid pipelines can approach large language model accuracy at a fraction of the cost. We fine-tune four open-source encoders (IndoBERT-base, IndoBERT-large with Low-Rank Adaptation, IndoBERTweet, and IndoBERT-lite), evaluate four commercial large language model arms (Claude Haiku, Sonnet, and Opus under zero-shot and few-shot prompting), and build three hybrid pipelines—confidence routing, uncertainty cascading, and knowledge distillation—on four Indonesian sentiment datasets (SmSA, NusaX, UGM Twitter, and ID-SMSA). Accuracy is measured by Macro-F1 over three seeds, cost by a per-1,000-prediction model, and differences by McNemar's test with bootstrap confidence intervals. Paying for a large language model is statistically significant on only one of four datasets: on SmSA the best open-source encoder reaches 0.924 against 0.966 for Sonnet, whereas on the noisy UGM corpus the free encoder (0.708) significantly outperforms every paid model (≤0.651, p<0.001). Confidence routing is the consistent winner, reaching 0.947 on SmSA at only US$0.18 per 1,000 predictions—about 98% of full-model accuracy at one-ninth of the cost—and surpassing both pure endpoints on three of four datasets. The contribution is a statistically grounded, reproducible decision framework for selecting sentiment pipelines in low-resource languages that specifies when paying for large language models is justified and when free fine-tuned encoders suffice.

Copyrights © 2026






Journal Info

Abbrev

jurnal

Publisher

Subject

Computer Science & IT

Description

Jurnal Teknik Informatika (JUTIF) is an Indonesian national journal, publishes high-quality research papers in the broad field of Informatics, Information Systems and Computer Science, which encompasses software engineering, information system development, computer systems, computer network, ...