RABIT: Jurnal Teknologi dan Sistem Informasi Univrab
Vol 11 No 2 (2026): Juli

EVALUASI SKEMA FUNGSI TERKOMPRESI PADA SMALL LANGUAGE MODEL MENGGUNAKAN QLORA PADA DOMAIN APARATUR SIPIL NEGARA

Alfan Lily Armansyah (Universitas Amikom Purwokerto)
Imam Tahyudin (Unknown)
Azhari Shouni Barkah (Unknown)



Article Info

Publish Date
10 Jul 2026

Abstract

As government institutions undergo digitalization, demand has grown for intelligent assistants that can resolve personnel inquiries autonomously by means of function calling. Commercial large language models remain difficult to deploy in this setting owing to recurring subscription fees and potential exposure of confidential records, which positions locally hosted Small Language Models (SLMs) as a viable substitute. However, conventional verbose JSON schemas exhaust the context window and degrade the accuracy of small models. This study proposes the Compressed Function Schema (CFS), a concise notation replacing verbose JSON schemas, combined with QLoRA fine-tuning on the Qwen3.5-0.8B model for the personnel domain of the Indonesian Civil Service (ASN). Evaluation was conducted on 173 test samples under three controlled conditions, namely baseline (verbose schema, zero-shot), ablation (compressed schema, zero-shot), and proposed (compressed schema, fine-tuned), using Function Name Accuracy (FNA), Parameter Accuracy (PA), and Exact Match Accuracy (EMA) metrics. Results show that CFS reduces schema tokens from 1,739 to 302 tokens, or 5.76 times more compact, a significant reduction, while the proposed model achieves FNA of 95.95%, PA of 97.20%, and EMA of 91.03%, outperforming the baseline across all metrics. The ablation study reveals that the compressed schema does not function without fine-tuning, indicating that the two are mutually complementary. Accordingly, an 0.8-billion-parameter SLM can operate on a 16 GB commodity GPU with high accuracy for personnel information services.

Copyrights © 2026






Journal Info

Abbrev

rabit

Publisher

Subject

Computer Science & IT Engineering

Description

This journal is called RABIT, where the name comes from two words namely, RAB which means Abdurrab University and IT which means information technology, it can be interpreted as a journal of this journal Journal of Informatics Engineering Study Program Pekanbaru Abdurrab University. This RABIT ...