Automated Tri Dharma Quality Assessment for Academic Accreditation Using Qwen, Mistral, and DeepSeek

Mochammad Ariel Sulton (Applied Data Science, Electronic Engineering Polytechnic Institute of Surabaya, Indonesia)
Tita Karlita (Applied Data Science, Electronic Engineering Polytechnic Institute of Surabaya, Indonesia)
Nyoman Bayu Surapati (Game Technology, Electronic Engineering Polytechnic Institute of Surabaya, Indonesia)
Firnanda Pristiana Nurmaida (Game Technology, Electronic Engineering Polytechnic Institute of Surabaya, Indonesia)
Faros Alaudin Althaf (Power Plant Engineering, Electronic Engineering Polytechnic Institute of Surabaya, Indonesia)
Yesta Medya Mahardhika (Informatics Engineering, Electronic Engineering Polytechnic Institute of Surabaya, Indonesia)
Paramita Eka Wahyu Lestari (Telecommunication Engineering, Electronic Engineering Polytechnic Institute of Surabaya, Indonesia)
Aris Bahari Rizki (Telecommunication Engineering, Electronic Engineering Polytechnic Institute of Surabaya, Indonesia)

Equilibrium: Jurnal Pendidikan
Vol. 14 No. 2 (2026): EQUILIBRIUM: JURNAL PENDIDIKAN

Internal Quality Audit (AMI) is critical for Indonesian higher-education accreditation, yet study programs still score 60+ indicators manually by cross-referencing a 200-page Self-Evaluation Report (LED) and a 50-sheet Study Program Performance Report (LKPS), a process that takes 3–5 days per program and varies between assessors. This study develops an automated quality-assessment agent that ingests both documents and benchmarks three open-weight Large Language Models, Qwen 3.5 35B, DeepSeek-R1 32B, and Mistral-Small 3.2 24B, on 19 LAM-Teknik indicators sampled across all nine accreditation criteria (9 qualitative, 6 quantitative, 4 composite). The reference scores are taken from the certified human assessor's official record produced during the program's 2024 post-audit cycle. The results show that Qwen 3.5 attains the lowest MAE (0.605) and RMSE (0.769) with 100% within ±1 accuracy and 32.5 s per indicator; DeepSeek-R1 is the most cautious but the slowest (MAE 1.395; 79.9 s) and collapses on quantitative items (MAE 2.17); Mistral is the fastest (13.6 s).