Journal of Language, Literature, and Educational Research
Vol. 3 No. 1 (2026): June

Exploring Rater Perceptions in Argumentative Writing Assessment: A Qualitative Case Study of Experience, Nativeness, and Rhetorical Variability

Gül KARAAGAC ZAN (Nigde Omer Halisdemir University)



Article Info

Publish Date
29 Jun 2026

Abstract

Purpose of the study: This study aims to explore how linguistic background and teaching experience influence evaluation criteria, rhetorical interpretation, and scoring decisions during assessment of argumentative writing. It also examines how raters perceive rhetorical variation in essays written by students from different cultural backgrounds. Methodology: This study employed an exploratory qualitative case study design using Concurrent Think-Aloud Protocols (TAP). Participants consisted of four English as a Foreign Language instructors with different linguistic backgrounds and teaching experience. Data were collected through audio-recorded assessment sessions and analyzed using structural coding and pattern coding to identify recurring evaluation themes and scoring behaviors. Main Findings: Findings revealed that native English-speaking raters emphasized coherence, audience awareness, and rhetorical organization, whereas non-native English-speaking raters focused primarily on grammatical accuracy and content. Rhetorical variation influenced evaluation decisions, producing differences in assigned scores across participants. Teaching experience and linguistic background contributed to variation in assessment criteria, rhetorical interpretation, and scoring practices during evaluation of argumentative writing. Novelty/Originality of this study: This study integrates linguistic nativeness, teaching experience, and rhetorical variation within one qualitative investigation using Think-Aloud Protocols. Unlike previous studies emphasizing score comparison, it explores cognitive processes underlying writing assessment decisions. Findings provide deeper understanding of how rater characteristics shape evaluation practices and support development of more inclusive and reliable writing assessment in multilingual educational contexts.

Copyrights © 2026






Journal Info

Abbrev

JoLLE

Publisher

Subject

Humanities Education Languange, Linguistic, Communication & Media Social Sciences Other

Description

Journal of Language, Literature, and Educational Research is a scientific journal published by Cahaya Ilmu Cendekia Publisher. This journal distributes papers based on studies and literature reviews in Indonesian, Regional, and Foreign Education for primary, secondary, senior, and higher education. ...