Transformative Practices: Integrating Automated Writing Evaluation in Higher Education Writing Classrooms - A Systematic Review - Indri Astutik; Utami Widiati; Devinta Puspita Ratri; Peggy Magdalena Jonathans; Nurkamilah Nurkamilah; Yeni Mardiyana Devanti; Zaldi Harfal (Indonesian Journal on Learning and Advanced Education (IJOLAE), 2024)
This systematic literature review explores the utilization of Automated Writing Evaluation (AWE) as a writing scoring tool over a five-year period from 2016 to 2020, focusing on its role in the transformation and integration of learning tools for pedagogical purposes. Transformation refers to the significant changes and advancements in teaching methods, particularly in adapting to new educational technologies and approaches, while integration involves the seamless incorporation of AWE systems into these evolving instructional practices to enhance the effectiveness of writing instruction. The study aims to analyze the various types of AWE employed in academic research, track trends in AWE technology strategies, and investigate students’ perceptions of AWE in both scoring and instructional contexts. Additionally, it aims to uncover the benefits and limitations associated with AWE implementation in writing instruction. Examining 19 journal articles, this review identifies fourteen types of AWE utilized by researchers and tracks advancements in machine learning within the field. The findings reveal positive student perceptions of AWE, citing its usefulness, efficiency, and linguistic accuracy in scoring and instruction. Benefits of AWE implementation include improved linguistic accuracy, enhanced writing performance, increased student engagement, and the provision of reliable and valid feedback. Moreover, AWE demonstrates effectiveness in scoring and feedback provision, with potential short- and long-term effects on student learning. However, limitations of AWE are also noted, including student distrust of feedback and a preference for human raters over AWE-generated scores. This review provides valuable insights into the multifaceted role of AWE in writing instruction, highlighting its potential benefits and areas for improvement.