Arranging a writing test without considering students’ level of understanding of the test items may lead to misunderstandings during the test, particularly in a class with diverse writing abilities. This research aims to explore students’ insights to enhance the quality of writing tests as an attempt to avoid misunderstandings regarding writing test questions and instructions by utilising readability tests. The assessment was conducted at a private Islamic secondary school, where students exhibited a range of writing abilities. A total of 19 students participated in this evaluation. Data were collected through the following process: (1) students read and completed the writing test; (2) they filled out a readability test to identify their comprehension of the writing test questions and instructions; (3) the gathered data were classified into three categories (writing comprehension test, writing skill test, and test suitability). The results indicated that six students did not understand the writing test in Sections A and B due to their limited vocabulary. However, they demonstrated a better understanding of the writing test instructions (Section C), even though they had not fully utilised the provided vocabulary list for writing. Overall, the students felt that the writing test was suitable for their level of learning. In summary, readability assessments can serve as an alternative means to identify varied students’ understanding of writing test questions and instructions. This finding is beneficial for teachers in designing writing tests that accommodate students with varying writing competencies. Given that many students struggle with comprehending writing questions and instructions, teachers can assist this process by providing translations of the instructions.