Journal of Artificial Intelligence and Digital Business
Vol. 5 No. 2 (2026): Mei-Juli

Automated MSME Bookkeeping Using Vision-Language Models and Hierarchical Multi-Agent Systems

La Ode Muhammad Yudhy Prayitno (Halu Oleo University)
La Ode Muhammad Golok Jaya (Halu Oleo University)
Asa Hari Wibowo (Halu Oleo University)



Article Info

Publish Date
26 Jun 2026

Abstract

We present a bookkeeping automation system for micro, small, and medium enterprises (MSMEs) that converts receipt photographs into structured ledger entries without manual data transcription. Receipt-based automation is difficult because document layouts vary widely across retailers, image quality in real-world capture conditions is unreliable, and writing extracted data into a spreadsheet accounting system requires coordinated multi-step execution that a single model cannot handle end to end. The system fine-tunes Qwen 3.5-4B with Low-Rank Adaptation (LoRA) on a 1,000-image annotated receipt corpus and couples the extraction model to a LangGraph-based hierarchical multi-agent architecture whose agents access SQLite and Google Sheets through standardized Model Context Protocol (MCP) tool endpoints. When evaluated on 100 unseen test images, the fine-tuned model, configured at LoRA rank 32 and epoch 3, yields a 94.58% Digit Accuracy and an ANLS* score of 93.99 while maintaining 100% JSON validity, representing a 43.71-point and 17.09-point improvement in KIEVal Group F1 and KIEVal Entity F1, respectively, over the untuned Qwen 3.5-4B baseline and outperforming all four zero-shot baselines across every metric. Beyond individual model capabilities, the multi-agent framework, served through vLLM and exposed via a cross-platform mobile client, successfully satisfies all 22 behavioral test cases covering single-agent tool routing, cross-agent orchestration, human-in-the-loop confirmation, and input-output guardrail enforcement, demonstrating a deployable end-to-end pipeline from receipt capture to ledger update.

Copyrights © 2026






Journal Info

Abbrev

RIGGS

Publisher

Subject

Computer Science & IT Economics, Econometrics & Finance Electrical & Electronics Engineering Engineering

Description

Journal of Artificial Intelligence and Digital Business (RIGGS) is published by the Department of Digital Business, Universitas Pahlawan Tuanku Tambusai in helping academics, researchers, and practitioners to disseminate their research results. RIGGS is a blind peer-reviewed journal dedicated to ...