Yasin, Suhaila Mohd.
Unknown Affiliation

Published : 2 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 2 Documents
Search

Student-Generated User Story Quality: A Study on Practitioner and ChatGPT Evaluation Zul, Muhammad Ihsan; Yasin, Suhaila Mohd.; Sahid, Dadang Syarif Sihabudin
Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) Vol 9 No 5 (2025): October 2025
Publisher : Ikatan Ahli Informatika Indonesia (IAII)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29207/resti.v9i5.6950

Abstract

Evaluating the quality of student-generated user stories is important in software engineering education, but only a limited number of industry practitioners can assist. The integration of generative AI can facilitate this process. To do so, the INVEST quality evaluation framework is widely recognized for assessing user story quality; however, prior research has not explored its use in conjunction with generative AI. This study investigated ChatGPT's ability to evaluate user stories using the INVEST framework. This study compares two ChatGPT-based evaluation approaches with those of experienced practitioners, focusing on student-generated user stories. Discrepancies between ChatGPT and practitioner evaluations were measured using Mean Absolute Deviation (MAD), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE). Statistical significance was tested using the Mann-Whitney U Test. The results indicate that ChatGPT’s 1st approach yielded lower discrepancies than practitioner evaluations. Moreover, significance testing showed no statistically significant differences between the ChatGPT and practitioner results for the two INVEST criteria- Independent and Estimable. These findings suggest that the 1st approach can assist in the evaluation process, although practitioners must ensure comprehensive and accurate evaluations. ChatGPT can provide preliminary evaluations in educational contexts, enabling students to receive formative feedback and allowing educators to streamline evaluation processes. Although practitioner validation is still required, their role may shift toward verifying AI-generated results, thus reducing the overall workload and accelerating quality evaluation
An Enhanced Model for Evaluating Learning Satisfaction in Teaching User Stories: A Confirmatory Factor Analysis Approach Zul, Muhammad Ihsan; Yasin, Suhaila Mohd.; Sahid, Dadang Syarif Sihabudin
Journal of Information Systems Engineering and Business Intelligence Vol. 12 No. 1 (2026): February
Publisher : Universitas Airlangga

Show Abstract | Download Original | Original Source | Check in Google Scholar

Abstract

Background: Understanding how students experience and perceive learning through user stories provides valuable insights into the effectiveness of instructional design. Huang proposed a learning satisfaction framework in which students’ satisfaction emerges from four factors, namely perceived ease of use (PEOU), perceived usefulness (PU), learning motivation (PM), and overall learning satisfaction (LS). A recent study applied this model to teaching user stories in a software engineering course using Confirmatory Factor Analysis (CFA) and Partial Least Squares Structural Equation Modeling (PLS-SEM) but reported suboptimal model fit, with unsatisfactory SRMR and NFI values, indicating the need for refinement. Objective: This study aims to evaluates an enhanced learning satisfaction model for teaching user stories by identifying key influencing factors, examining their relationships, and assessing construct validity and model fit improvements. Methods: To improve the model, additional theoretical paths were introduced, and survey data were collected from 142 randomly selected software engineering students. The refined model was evaluated using CFA and PLS-SEM, with model fit improvements assessed through SRMR and NFI values. Results: The analysis revealed that PEOU and LM significantly influence learning satisfaction, while PU affects satisfaction indirectly through motivation. These interactions among PU, PEOU, and LM explain how ease of use and usefulness enhance motivation, which in turn increases satisfaction. Furthermore, the enhanced model showed an improved fit compared to the previous version, with SRMR values decreasing from 0.092 to 0.076 and NFI improving from 0.765 to 0.813, confirming better construct validity and overall model fit. Conclusion: The addition of new direct paths from PEOU and PU to LS increased the model’s R² and Q² values, indicating stronger construct validity and better overall fit. The refined structure provides a more accurate representation of how satisfaction is formed and offers a validated instrument for evaluating student learning experiences in teaching user stories within software engineering course.   Keywords: learning satisfaction, user story, confirmatory factor analysis, model fit evaluation, PLS-SEM, software engineering education.