Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal IJID (International Journal on Informatics for Development)
Aulia Faqih Rifa’i
Informatika UIN Sunan Kalijaga Yogyakarta
Computer Science & IT

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Development and Post-Hoc Evaluation of a Transformer-Based Mental Health Chatbot Using the ESPRT Framework David S; Muhammad Syafiq Akmal; Aulia Faqih Rifa’i
IJID (International Journal on Informatics for Development) Vol. 15 No. 1 (2026): IJID JUNE
Publisher : Faculty of Science and Technology, Universitas Islam Negeri (UIN) Sunan Kalijaga Yogyakarta

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.14421/ijid.2026.6024

Abstract

Mental health disorders are an increasing global concern, yet access to professional care is still limited due to resource shortages and persistent social stigma. Artificial intelligence–based chatbots have therefore emerged as a potential tool for providing early support. However, developing systems that are both computationally efficient and psychologically appropriate remains challenging. This study developed a chatbot using the FLAN-T5-base architecture, fine-tuned on the MentalChat16K dataset, comparing Full Fine-Tuning (FFT) and Low-Rank Adaptation (LoRA). Quantitative evaluation yielded objectively low n-gram scores (e.g., ROUGE-L of 17.81 for FFT and 17.17 for LoRA). Although these scores may suggest weak generative performance, they were largely affected by the brevity penalty because the models generated much shorter responses than the reference answers. Thus, to further evaluate response quality, an LLM-as-a-judge qualitative assessment was conducted, confirming that both models produced safe, contextually relevant, and reasonably empathetic responses. Notably, LoRA achieved comparable qualitative performance while updating only ~0.35% of parameters, reducing GPU memory usage by 21% and training time by about 24%. Interpreted through the Eco-Socio-Psycho-Religio-Technic (ESPRT) post-hoc discussion framework, the findings demonstrate that LoRA is a practical and sustainable approach for mental health support systems in resource-limited environments.
Co-Authors David S Muhammad Syafiq Akmal