Comparing IRT Models: Summated Scaling Effects on Critical Thinking in Vocational Students - Manggaberani, Andi Abdurrahman; Samsul Hadi; Nur Hidayanto Pancoro Setyo Putro; Fajri, Abrar Syahrul; Retnawati, Heri (JP3I (Jurnal Pengukuran Psikologi dan Pendidikan Indonesia), 2025)
This study investigates the comparative efficacy of Summated Rating Scales (SRS) and traditionalordinal scales (raw Likert-type responses) in measuring critical thinking skills among vocationalstudents, employing Item Response Theory (IRT) to evaluate their psychometric properties.Addressing the limitations of ordinal scales notably inconsistent intervals between responsecategories the research adopts a descriptive quantitative methodology involving 269 students fromstate vocational high schools in Yogyakarta, Indonesia. Data were collected using a five-pointLikert scale instrument, validated for content (Aiken’s V = 0.94), and analyzed through two IRTframeworks: Polytomous IRT for unscaled ordinal data and Continuous Response Model (CRM)IRT for SRS-transformed interval data. Key findings reveal that SRS enhances measurementprecision by normalizing response distributions into proportional intervals (e.g., recalibrated scores:0.00, 0.73, 1.46, 2.07, 2.84), thereby resolving issues of unequal category spacing inherent toordinal scales. Polytomous IRT demonstrated robust item fit (e.g., Partial Credit Model fit for 5/6items) and strong difficulty parameter invariance (r = 0.84), yet exhibited instability in abilityestimates (r = 0.37) due to extreme response patterns. Conversely, CRM IRT applied to scaleddata produced stable ability estimates (r = 0.46) and eliminated infinite values in MaximumLikelihood Estimation, underscoring its superiority in handling continuous metrics. However, ordinalscales retained higher consistency in difficulty calibration across subgroups. The study concludesthat integrating SRS with CRM IRT offers a refined approach for critical thinking assessments,balancing precision and fairness, while ordinal scales remain pragmatic for contexts prioritizingsimplicity. These insights advocate for the adoption of advanced scaling techniques in vocationaleducation to improve the validity of competency evaluations, with recommendations for futureresearch to explore hybrid models and longitudinal applications.