Article Per Year (5 Year)
p-Index From 2021 - 2026
3.082
P-Index
This Author published in this journals
All Journal LET: Linguistics, Literature and English Teaching Journal Vocatech : Vocational Education and Technology Journal Jurnal Vokasi Jurnal Ilmiah Manusia dan Kesehatan Jurnal Ilmiah Kanderang Tingang AVERROUS: Jurnal Kedokteran dan Kesehatan Malikussaleh Jurnal Ilmu Sosial dan Ilmu Politik Malikussaleh (JSPM) Jurnal Sosiologi Dialektika Sosial Kande : Jurnal Ilmiah Pendidikan Bahasa dan Sastra Indonesia Nanggroe: Journal Of Scholarly Service Jurnal Pengabdian Masyarakat Bangsa Nanggroe: Journal of Scholarly Service Socius: Social Sciences Research Journal Jurnal Malikussaleh Mengabdi Jurnal Solusi Masyarakat (JSM) Jurnal Solusi Masyarakat Dikara
Teuku Azhari, Teuku
Unknown Affiliation
Religion Agriculture, Biological Sciences & Forestry Arts Humanities Chemical Engineering, Chemistry & Bioengineering Chemistry Civil Engineering, Building, Construction & Architecture Computer Science & IT Decision Sciences, Operations Research & Management Dentistry Economics, Econometrics & Finance Education Electrical & Electronics Engineering Energy Engineering Environmental Science Health Professions Industrial & Manufacturing Engineering Languange, Linguistic, Communication & Media Law, Crime, Criminology & Criminal Justice Library & Information Science Materials Science & Nanotechnology Mathematics Mechanical Engineering Medicine & Pharmacology Neuroscience Nursing Physics Public Health Social Sciences Other

Published : 21 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 21 Documents
Search

 1   2   3 
Validating an Automated Essay Evaluation System for Vocational EFL Learners Using Icnale-Gra and a CEFR-J Writing Level Analyzer (CWLA) Mahlil, Mahlil; Masykar, Tanzir; Azhari, Teuku
VOCATECH: Vocational Education and Technology Journal Vol 8, No 1 (2026): April
Publisher : Akademi Komunitas Negeri Aceh Barat

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.38038/vocatech.v8i1.315

Abstract

AbstractThis paper focuses on the authenticity of an automated essay grader in determining English writing competence in vocational higher learning students in an English as a Foreign Language (EFL) setting. The study was done in Politeknik Negeri Lhokseumawe, Indonesia where first-year second-semester students of the Information and Communication Technology (ICT) Department participated. The primary dataset consisted of 40 student essays that were gathered at the final writing examination. The essays were graded with the use of an AI-based essay grading through the help of an essay grading tool of Writecream (app.writecream.com), which would provide automated writing scores on the basis of the linguistic and structural characteristics. In an attempt to authenticate the validity of the AI scoring results, this study has used two benchmarking frameworks. The International Corpus Network of Asian Learners of English (ICNALE Online) with special attention to the ICNALE-GRA database was utilized as a corpus-based reference, and it represented the standardized levels of writing proficiency of learners. Second, the Writing Level Analyzer (CWLA) based on CEFR was used where the essays were categorized into the bands of the writing proficiency on the CEFR-J framework. The use of CVLA3 desktop version assisted in the classification of the essays into those bands. The validation process was conducted against the AI-generated scores against CEFR-J level classifications and corpus-based proficiency standards. The results showed that the automated essay assessment system showed a validity rate of 62% which showed moderate level of validity with the results of the CEFR-J aligned assessment. This implies that AIs scoring devices can be of service when it comes to vocational EFL formative writing assessment. Nevertheless, the moderate validity raises the importance of using automated systems alongside human judgment particularly in high-stakes measures as with final examination. Altogether, the research adds practice to the existing body of research on AI-aided language testing and underlines the relevance of validation based on CEFR in education. Keywords:AES; CEFR-J; CWLA; ICNALE-GRA corpus; EFL writing assessment. AbstrakPenelitian ini berfokus pada otentisitas penilai esai otomatis dalam menentukan kompetensi menulis bahasa Inggris pada mahasiswa pendidikan tinggi vokasional dalam konteks Bahasa Inggris sebagai Bahasa Asing (EFL). Studi ini dilakukan di Politeknik Negeri Lhokseumawe, Indonesia, di mana mahasiswa semester dua tahun pertama Jurusan Teknologi Informasi dan Komunikasi (TIK) berpartisipasi. Dataset utama terdiri dari 40 esai mahasiswa yang dikumpulkan pada ujian menulis akhir. Esai-esai tersebut dinilai dengan menggunakan penilaian esai berbasis AI melalui bantuan alat penilaian esai Writecream (app.writecream.com), yang akan memberikan skor menulis otomatis berdasarkan karakteristik linguistik dan struktural. Dalam upaya untuk mengotentikasi validitas hasil penilaian AI, studi ini menggunakan dua kerangka kerja benchmarking. International Corpus Network of Asian Learners of English (ICNALE Online) dengan perhatian khusus pada basis data ICNALE-GRA digunakan sebagai referensi berbasis korpus, dan mewakili tingkat kemampuan menulis standar para pembelajar. Kedua, Writing Level Analyzer (CWLA) berbasis CEFR digunakan di mana esai dikategorikan ke dalam tingkatan kemampuan menulis pada kerangka CEFR-J. Penggunaan versi desktop CVLA3 membantu dalam klasifikasi esai ke dalam tingkatan tersebut. Proses validasi dilakukan terhadap skor yang dihasilkan AI terhadap klasifikasi tingkat CEFR-J dan standar kemampuan berbasis korpus. Hasil menunjukkan bahwa sistem penilaian esai otomatis menunjukkan tingkat validitas 62% yang menunjukkan tingkat validitas moderat dengan hasil penilaian yang selaras dengan CEFR-J. Ini menyiratkan bahwa perangkat penilaian AI dapat bermanfaat dalam hal penilaian penulisan formatif EFL kejuruan. Meskipun demikian, validitas moderat meningkatkan pentingnya penggunaan sistem otomatis bersamaan dengan penilaian manusia, khususnya dalam pengukuran berisiko tinggi seperti ujian akhir. Secara keseluruhan, penelitian ini menambah praktik pada penelitian yang ada tentang pengujian bahasa yang dibantu AI dan menggarisbawahi relevansi validasi berbasis CEFR dalam pendidikan..
Co-Authors Amanda, Lola Deddy Satria M Dewi Kumala Sari Fajriana, F Fazilah, Nurul Fidyati, Fidyati Harliyana, Iba Idaryani Idaryani Idaryani Juhera, Juhera Juwita Sahputri Kurniawati Kurniawati Mahlil, Mahlil Masykar, Tanzir Millizia, Anna Mukhlis Mukhlis Mursalin, M Nabila, Putri Nisa, Zahratul Hadil Pratiwi, Rani Ardesi Pulungan, Alyifia Yulia Maura Rasyimah Reza Pahlevi Ginting Rizka, Adi Rizki, Dini Rohantizani, R Sahputri, Juwita Sarah Rahmayani Siregar Wizar Putri Mellaratna Wulanda