Validity and Reliability of Science Literacy Assessment Instruments for Measuring Science Competencies in the Context of PISA 2025 using the Rasch Model - Putri, Mentari Darma; Fitria, Dini; Nurlaini; Berutu, Nur Jannah (Jurnal Pendidikan Fisika, 2026)
Science literacy is a key competency for preparing students to reason scientifically, interpret evidence, and make informed decisions in real-world contexts. Yet Indonesian students’ performance in international assessments remains below the OECD average, underscoring the urgent need for assessment tools aligned with global standards. This study aimed to develop and validate a PISA 2025–oriented science literacy assessment instrument to support the measurement of senior high school students’ competencies in scientifically explaining phenomena; constructing and evaluating investigation designs; critically interpreting data and evidence; and researching, evaluating, and using scientific information for decision-making and action. Using a research and development approach with the 4-D model, ten context-rich essay items and an analytic scoring rubric (levels 1–4) were produced. Content validity was examined by three experts using Aiken’s V, and empirical validation was conducted with 50 Year 11 students from two senior high schools in Langsa City using the Rasch Model (Winstep). The instrument demonstrated strong content validity (Aiken’s V averages: construction = 0.84, relevance = 0.89, clarity = 0.90). Rasch results showed that 8 of 10 items met fit criteria, while two items displayed misfit on selected indices but remained positively correlated with the measured construct. Reliability was high (Cronbach’s alpha = 0.85; person reliability = 0.84; item reliability = 0.97), and item difficulty spanned a broad range, enabling discrimination across students’ ability levels. The novelty of this work lies in producing a Rasch-validated, constructed-response instrument explicitly aligned with the revised PISA 2025 science competency structure, including the new competency on evaluating and using scientific information in digital contexts. In conclusion, the instrument is suitable for limited classroom and research use. It provides a practical contribution to physics education by strengthening evidence-based assessment of scientific reasoning and decision-making in physics-related and environmental contexts.