Comparative Analysis of the Effectiveness of Informatics Course Learning Utilizing Chatgpt - Utami, Arneitta Dwicahya; Kamayani, Mia; Siduningrum, Estu; Azhar, Nur Chalik (Acta Pedagogia Asiana, 2025)
This study examined the effectiveness of conventional teaching methods and ChatGPT in an introductory Algorithms and Programming course at the university level. ChatGPT, an AI-based NLP technology, assisted students in understanding course material through automated responses. However, its effectiveness relative to conventional methods required further evaluation, particularly concerning motivation, interaction, self-regulation, instructional structure, and the instructor's role. Using a sample of 10 students for pretest-posttest analysis, 38 respondents for the User Experience Questionnaire (UEQ), and accuracy analysis via prompt engineering, the results revealed that conventional methods better enhanced motivation and interaction. ChatGPT demonstrated strengths in attractiveness (1.982) and efficiency (2.053) but scored lower in accuracy (1.395) and novelty (1.053). Prompt engineering significantly improved response accuracy when tailored to learning modules, highlighting the importance of precise inputs. The findings suggested that while ChatGPT excelled as a supplementary tool, it was less effective as a standalone teaching method. This study contributed to the growing field of educational technology by providing insights into the integration of AI tools in learning environments.