Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)
Yasin, Suhaila Mohd.
Unknown Affiliation
Computer Science & IT Engineering

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Student-Generated User Story Quality: A Study on Practitioner and ChatGPT Evaluation Zul, Muhammad Ihsan; Yasin, Suhaila Mohd.; Sahid, Dadang Syarif Sihabudin
Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) Vol 9 No 5 (2025): October 2025
Publisher : Ikatan Ahli Informatika Indonesia (IAII)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29207/resti.v9i5.6950

Abstract

Evaluating the quality of student-generated user stories is important in software engineering education, but only a limited number of industry practitioners can assist. The integration of generative AI can facilitate this process. To do so, the INVEST quality evaluation framework is widely recognized for assessing user story quality; however, prior research has not explored its use in conjunction with generative AI. This study investigated ChatGPT's ability to evaluate user stories using the INVEST framework. This study compares two ChatGPT-based evaluation approaches with those of experienced practitioners, focusing on student-generated user stories. Discrepancies between ChatGPT and practitioner evaluations were measured using Mean Absolute Deviation (MAD), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE). Statistical significance was tested using the Mann-Whitney U Test. The results indicate that ChatGPT’s 1st approach yielded lower discrepancies than practitioner evaluations. Moreover, significance testing showed no statistically significant differences between the ChatGPT and practitioner results for the two INVEST criteria- Independent and Estimable. These findings suggest that the 1st approach can assist in the evaluation process, although practitioners must ensure comprehensive and accurate evaluations. ChatGPT can provide preliminary evaluations in educational contexts, enabling students to receive formative feedback and allowing educators to streamline evaluation processes. Although practitioner validation is still required, their role may shift toward verifying AI-generated results, thus reducing the overall workload and accelerating quality evaluation
Co-Authors Dadang Syarif Sihabudin Sahid Muhammad Ihsan Zul