Content Validity and Inter-Rater Reliability in Assessing Environmental Literacy and Creative Thinking Skills of Students - Anderias Henukh; Agus Setiawan; Lilit Rusyati; Dadi Rusdiana; Yusri Kamin; Tutik Yuliatun (Jurnal Pendidikan Sains Indonesia, 2025)
This study evaluates the need for validated instruments to measure students' environmental literacy (EL) and creative thinking skills (CTS). It specifically aims to evaluate the content validity and inter-rater reliability of these instruments. Content validity was evaluated using Aiken's V, which quantified the level of agreement among three experts on the relevance and suitability of instrument items. The instrument includes 30 questions for EL, and 16 questions for CTS. Subsequently, inter-rater reliability was analyzed with Cohen's Kappa and Fleiss' Kappa to measure agreement between two or more raters. The findings demonstrate that both the EL and CTS instruments possess excellent content validity, with the majority of items rated as valid by the experts. Furthermore, the analysis indicated moderate to high agreement among the raters, confirming the instruments' reliability for continued research. Future research is encouraged to incorporate a greater number of experts for content validity testing to gain a more diverse perspective, as well as more raters in the reliability analysis to enhance the representation of agreement