Despite the central role of teachers in fostering students’ scientific literacy, assessment instruments targeting in-service science teachers remain limited. This study aimed to develop and examine the preliminary quality of a scientific literacy instrument for junior high school science teachers, informed by OECD documents outlining the direction of the PISA 2025. A design-based research approach was employed using Tessmer’s formative evaluation model, encompassing iterative stages of preliminary analysis to pilot field test. The instrument comprised nine context-rich stimuli and 25 items across multiple formats designed to elicit scientific reasoning and decision-making in real-world contexts. Expert validation produced a Mean Expert Score of 4.67, while teacher readability evaluation yielded a Mean Readability Score of 4.29, indicating strong content representation and clarity. Pilot psychometric analysis showed a balanced distribution of item difficulty (32% easy, 40% moderate, and 28% difficult), and item-level validity and discrimination indices provided diagnostic evidence for refinement. Pilot psychometric analysis indicated that the instrument’s reliability remains preliminary and highlights the need for further refinement through larger-scale field testing. Overall, this study contributes a forward-looking assessment instrument that supports formative evaluation of teachers’ scientific literacy and provides a robust foundation for subsequent large-scale validation aligned with emerging international assessment orientations.
Copyrights © 2025