Establishing the Validity and Reliability of a Program Evaluation Questionnaire using Rasch Measurement Model - Loh, Su Ling; Jamil, Nur Aisah; Thoe, Ng Khar (Universitas Muhammadiyah Purwokerto, 2022)
This study was conducted to establish the validity and reliability of the evaluation questionnaire for the 12th Regional Congress of the Search For SEAMEO Young Scientist (SSYS) 2022 using the Rasch Measurement Model that was aided by the Winsteps software. The questionnaire contains 24 items that evaluate the Congress's objectives, inputs, as well as event management and administration. Each item is rated on a 4-point rating scale. The instrument was administered at the end of the 3-days SSYS Congress held virtually in which 1891 participants submitted their responses. The establishment of validity and reliability of this questionnaire is crucial before further analysis is carried out. The Rasch Model analysis showed that the reliability index of the respondents was 0.87 and person separation is 2.60, while the item reliability index is 0.96 with an item separation index of 5.08. Item polarity indicates that the point measure correlation (PTMEA CORR) for the 24 items is between 0.67 to 0.76. In terms of item fit, the results indicated one misfit item that needs improvement in the future. The Principal Component Analysis (PCA) shows that almost all the items are unidimensional and intended to measure a similar trait. All these indicate the reliability of the questionnaire, and researchers can proceed with further data analysis to evaluate the 12th Regional Congress of the SSYS 2022.