Gated, grounded, and governed: integrating large language models, authorised tool calling, and retrieval-augmented generation into NeoSiakad.com, a multi-tenant academic information SaaS

M. Najamudin Ridha (Politeknik Negeri Tanah Laut)

Jatilima : Jurnal Multimedia Dan Teknologi Informasi
Vol. 7 No. 07 (2026): Jatilima : Jurnal Multimedia Dan Teknologi Informasi

Large language models (LLMs) are attractive front-ends for academic information systems, yet a campus assistant must not invent grades, expose other students’ records, or drift into general chat paid from the institution’s budget. This study reports the design and empirical evaluation of an LLM assistant embedded in NeoSiakad.com, a multi-tenant academic information system delivered as software as a service to Indonesian higher-education institutions. Following a design science approach, the artefact combines four controls: a master control plane that provisions one gateway key with a USD ceiling per tenant, a fail-closed intent gate that classifies every message before the main model is called, read-only tool calling that executes canonical academic queries under the caller’s own permissions and tenancy, and retrieval-augmented generation (RAG) with mandatory citations and an abstain rule. Because RAG is not yet deployed, it was simulated on a 36-chunk academic guideline corpus. Evaluation used the public demo tenant, production telemetry, and seven language models accessed through the OpenRouter gateway. The intent gate reached 90.0 to 100 percent accuracy across models; the cheapest model let no disallowed message through at USD 0.04 per thousand calls. On the live tenant, 16 of 17 value-bearing turns reproduced the exact figures returned by the canonical resolvers, two cross-user probes were refused, and four disallowed prompts were rejected in 0.9 to 2.3 s without a main-model call. Retrieval reached Recall@3 of 1.00 for all five embedding models, and RAG raised fully correct answers from 0 to 14 percent (closed book) to 96 to 100 percent with 100 percent valid citations, while at most one of seven unanswerable questions was answered instead of declined. Moving chat turns to a queue worker cut the median turn latency from 40.9 s to 13.5 s on shared hosting. The results show that an LLM assistant can be made accurate, authorisation-bound, and cost-bounded on commodity infrastructure.