Jurnal Ilmiah Multidisiplin Indonesia
Vol. 5 No. 09 (2026): Jurnal Ilmiah Multidisplin Indonesia (JIM-ID), September 2026

LLM Token Consumption Optimization Using BERT Pre-filtering and Retrieval-Augmented Generation (RAG)

Nanang Kurnia Alfian (Master of Informatics Engineering, Universitas Amikom Yogyakarta)
Robert Marco (Master of Informatics Engineering, Universitas Amikom Yogyakarta)



Article Info

Publish Date
28 Sep 2026

Abstract

The adoption of Large Language Models (LLM) for software issue resolution causes token consumption inefficiency due to noisy and redundant context. This research proposes a hybrid system placing a BERT classification model as a context relevance pre-filter upstream of the Retrieval-Augmented Generation (RAG) pipeline, with hyperparameters optimized using Bayesian Optimization. A Design Science Research approach is applied through an ablation study of five scenarios (S0-S4) on GitHub Issues and SWE-bench datasets. The classification model achieves an F1-macro of 0.887, and BERT-based filtering (S2) reduces token consumption by 8.35% versus baseline, statistically significant (Wilcoxon, p<0.05). Bayesian Optimization yields a validation F1-macro of 0.889, higher than default hyperparameters. The main finding: token efficiency comes from the BERT pre-filter component, not from RAG which adds context.

Copyrights © 2026






Journal Info

Abbrev

esaprom

Publisher

Subject

Agriculture, Biological Sciences & Forestry Computer Science & IT Earth & Planetary Sciences Engineering Physics

Description

Jurnal Ilmiah Multidisiplin Indonesia (JIM-ID) is a peer-reviewed journal regularly published by the SEAN Institute every three months. namely, several research publications to publish multi-disciplinary articles with general topics on engineering, science, agriculture, plantations, forestry and ...