Claim Missing Document
Check
Articles

Found 3 Documents
Search

Individual ability on high-stakes test: Choosing cumulative score or rasch for scoring model Khair, Muhammad Dhiyaul; Marianti, Sukaesi
Jurnal Penelitian dan Evaluasi Pendidikan Vol. 28 No. 1 (2024)
Publisher : Graduate School, Universitas Negeri Yogyakarta in cooperation with Himpunan Evaluasi Pendidikan Indonesia (HEPI) Yogyakarta

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.21831/pep.v28i1.71661

Abstract

In a test, a method is required to estimate an individual's ability based on their responses. Typically, this is done by summing the correct responses or calculating a cumulative score. An alternative method is the Rasch model. This study aims to determine whether an individual's position, based on cumulative score estimates, remains unchanged or changes when compared with ability estimates using Rasch on dichotomous responses. The study uses open-source data from the 2018 Program for International Student Assessment (PISA) by the Organization for Economic Co-operation and Development (OECD) and involves 317 Indonesian students.  Ability analysis will be conducted on Math and Reading aspects using cumulative scores and Rasch with dichotomous responses. The study will employ data analysis techniques such as Rasch, paired samples t-test, and descriptive statistical analysis. The cumulative score and Rasch results will be tested using a paired samples t-test, and a comparison of the cumulative score and Rasch estimation results will be carried out using descriptive statistical analysis. The study results indicate that there are differences in individual positions based on ability estimates using cumulative score and Rasch. These differences are caused by variations in scores. Therefore, even if two individuals have the same cumulative score, they may have different Rasch estimates.
Comparing item-total correlation and item-theta correlation in test item selection: A simulation and empirical study Marianti, Sukaesi; Rufaida, Ana; Hasanah, Nur; Nuryanti, Sofia
Jurnal Penelitian dan Evaluasi Pendidikan Vol. 27 No. 2 (2023)
Publisher : Graduate School, Universitas Negeri Yogyakarta in cooperation with Himpunan Evaluasi Pendidikan Indonesia (HEPI) Yogyakarta

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.21831/pep.v27i2.61477

Abstract

One of the important processes in the evaluation of the psychometric properties of a test is item selection. The item selection process usually uses a very popular technique called item-total correlation. This study attempts to describe the item-total correlation technique and explore it using a similar technique called item-theta correlation. Both techniques are applied using simulation studies by creating several conditions related to test length and sample size. After the simulation study, the next step is the study using empirical data as an illustration of the results of the simulation study. The results of this study show that there are differences in the results of item selection based on these two approaches. Item-theta correlation detects more items that have weak discrimination power than item-total correlation. The difference is more noticeable in conditions where the cutoff point used for item selection is low(.20).
Evaluation of Different Person-Fit Measures in Cognitive Tests with Different Test Lengths Marianti, Sukaesi; Manao, Herdin Natalius; Faiha, Arij
JP3I (Jurnal Pengukuran Psikologi dan Pendidikan Indonesia) Vol 13, No 2 (2024): JP3I
Publisher : Fakultas Psikologi UIN Syarif Hidayatullah Jakarta

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.15408/jp3i.v13i2.34601

Abstract

Test takers’ characteristic is an exciting topic to discuss in psychometric research. In this study, person-fit is a part of the person characteristics applied in the context of cognitive tests. Given the importance of accurately estimating item and person parameters, person-fit is a statistical technique that can detect aberrant responses. Aberrance adversely affects the estimation process at the level of items and persons. The purpose of this study was to introduce and apply two popular person-fit statistics called and . These two statistics were applied in two studies, in study 1 using N = 317 and item = 16, and in study 2 using N = 331 and item = 49. The results showed that in studies 1 and 2,  detected more aberrant responses compared to . Significant differences in estimated results from both techniques were also shown in Study 2. The outcomes of this study are valuable for researchers and practitioners in the field of psychometrics who rely on , as a foundation for identifying aberrant responses.