Claim Missing Document
Check
Articles

Found 15 Documents
Search

Polytomous scoring correction and its effect on the model fit: A case of item response theory analysis utilizing R Santoso, Agus; Pardede, Timbul; Apino, Ezi; Djidu, Hasan; Rafi, Ibnu; Rosyada, Munaya Nikma; Retnawati, Heri; Kassymova, Gulzhaina K.
Psychology, Evaluation, and Technology in Educational Research Vol. 5 No. 1 (2022)
Publisher : Research and Social Study Institute

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.33292/petier.v5i1.148

Abstract

In item response theory, the number of response categories used in polytomous scoring has an effect on the fit of the model used. When the initial scoring model yields unsatisfactory estimates, corrections to the initial scoring model need to be made. This exploratory descriptive study used response data from Take Home Exam (THE) participants in the Statistical Methods I course organized by the Open University, Indonesia, in 2022. The stages of data analysis include coding the rater’s score; analyzing frequency; analyze the fit of the model based on graded, partial, and generalized partial credit models; analyze the characteristic response function (CRF) curve; scoring correction (rescaling); and re-analyze the fit of the model. The fit of the model is based on the chi-square test and the root mean square error of approximation (RMSEA). All model fit analyzes were performed by using R. The results revealed that scoring corrections had an effect on model fit and that the partial credit model (PCM) produced the best item parameter estimates. All results and their implications for practice and future research are discussed.
The effect of scoring correction and model fit on the estimation of ability parameter and person fit on polytomous item response theory Santoso, Agus; Pardede, Timbul; Djidu, Hasan; Apino, Ezi; Rafi, Ibnu; Rosyada, Munaya Nikma; Abd Hamid, Harris Shah
REID (Research and Evaluation in Education) Vol. 8 No. 2 (2022)
Publisher : Graduate School of Universitas Negeri Yogyakarta & Himpunan Evaluasi Pendidikan Indonesia (HEPI)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.21831/reid.v8i2.54429

Abstract

Scoring quality has been recognized as one of the important aspects that should be of concern to both test developers and users. This study aimed to investigate the effect of scoring correction and model fit on the estimation of ability parameters and person fit in the polytomous item response theory. The result of 165 students in the Statistics course (SATS4410) test at one of the universities in Indonesia was used to answer the problems in this study. The polytomous data obtained from scoring the test results were analyzed using the Item Response Theory (IRT) approach with the Partial Credit Model (PCM), Graded Response Model (GRM), and Generalized Partial Credit Model (GPCM). The effect of scoring correction and model fit on the estimation of ability and person fit was tested using multivariate analysis. Among the three models used, GRM showed the best fit based on p-value and RSMEA. The results of the analysis also showed that there was no significant effect of scoring correction and model fit on the estimation of the test taker's ability and person fit. From the results of this study, we recommend the importance of evaluating the levels or categories used in scoring student work on a test.
Gaining a deeper understanding of the meaning of the carelessness parameter in the 4PL IRT model and strategies for estimating it Pardede, Timbul; Santoso, Agus; Diki, Diki; Retnawati, Heri; Rafi, Ibnu; Apino, Ezi; Rosyada, Munaya Nikma
REID (Research and Evaluation in Education) Vol. 9 No. 1 (2023)
Publisher : Graduate School of Universitas Negeri Yogyakarta & Himpunan Evaluasi Pendidikan Indonesia (HEPI)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.21831/reid.v9i1.63230

Abstract

Three popular models are used to describe the characteristics of the test items and estimate the ability of examinees under the dichotomous IRT model, namely the one-, two-, and three-parameter logistic models. The three-item parameters are discriminating power, difficulty, and pseudo-guessing. In the development of the dichotomous IRT model, carelessness or upper asymptote parameter was proposed, which forms a four-parameter logistic (4PL) model to accommodate a condition where a high-ability examinee gives an incorrect response to a test item when he/she should be able to respond to the test item correctly. However, the carelessness parameter and the 4PL model have not been widely accepted and used due to several factors, and people's understanding of that parameter and strategies for estimating it is still inadequate. Therefore, this study aims to shed light on ideas underlying the 4PL model, the meaning of the carelessness parameter, and strategies used to estimate that parameter based on the extant literature. The focus of this study was then extended to demonstrating practical examples of estimating item and person parameters using the 4PL model using empirical data on responses of 1,000 students from the Indonesia Open University (Universitas Terbuka) on 21 of 30 multiple-choice items on the Business English test, a paper-and-pencil test. We mainly analyzed empirical data using the "˜mirt' package in RStudio. We present the analysis results coherently so that IRT users would have a sufficient understanding of the 4PL model and the carelessness parameter, and they can estimate item and person parameters under the 4PL model.
Penguatan Ketahanan Pangan Keluarga melalui Pelatihan Budidaya Sayuran Organik Berbasis Pemanfaatan Pekarangan Susi Sulistiana; Idha Farida; Mutiara Magta; Irla Yulia; Timbul Pardede
Jurnal Abdimas Kartika Wijayakusuma Vol 7 No 2 (2026): Jurnal Abdimas Kartika Wijayakusuma
Publisher : LPPM Universitas Jenderal Achmad Yani

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.26874/jakw.v7i2.1417

Abstract

Kegiatan Pengabdian kepada Masyarakat (PkM) ini bertujuan meningkatkan pengetahuan dan persepsi masyarakat mengenai ketahanan pangan keluarga yang dilakukan melalui pelatihan budidaya sayuran organik berbasis pemanfaatan pekarangan. Kegiatan ini dilaksanakan di wilayah Pondok Cabe Ilir dengan melibatkan 20 kader PKK sebagai peserta. Metode pelaksanaan meliputi observasi, pelatihan dan demonstrasi praktik budidaya, pembentukan tim pendampingan, serta evaluasi dengan menggunakan kuesioner pre-test dan post-test. Data dianalisis secara deskriptif komparatif untuk melihat adanya perubahan pengetahuan dan persepsi peserta sebelum dan sesudah dilakukan pelatihan. Hasil evaluasi menunjukkan bahwa terjadinya peningkatan pengetahuan peserta pada aspek manfaat sayuran organik dari 75% menjadi 95% serta terjadinya peningkatan pemahaman jenis sayuran yang dapat dipanen dari 45% menjadi 90%. Persepsi peserta terhadap manfaat sayuran organik juga mengalami perubahan yang positif terutama pada aspek kesehatan dan kontribusinya terhadap kebutuhan pangan keluarga. Kegiatan ini menunjukkan bahwa pelatihan berbasis praktik yang dilakukan dengan adanya pendampingan dinilai efektif dalam meningkatkan kapasitas masyarakat dalam memanfaatkan pekarangan sebagai sumber pangan keluarga di wilayah perkotaan.
Deep-Rasch as an Alternative to Rasch Modeling under Assumption Violations and Small Sample Sizes Agus Santoso; Farit Mochamad Afendi; Timbul Pardede; Heri Retnawati; Ibnu Rafi; Ezi Apino; Munaya Nikma Rosyada
CAUCHY: Jurnal Matematika Murni dan Aplikasi Vol 10, No 2 (2025): CAUCHY: JURNAL MATEMATIKA MURNI DAN APLIKASI
Publisher : Mathematics Department, Maulana Malik Ibrahim State Islamic University of Malang

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.18860/cauchy.v10i2.36276

Abstract

In certain situations, it may be challenging to fully exploit the advantages of modern test theory, including Rasch modeling and item response theory (IRT), when applied to real data. Although Rasch modeling tends to be more robust than IRT for small sample sizes, it still requires that the assumptions of unidimensionality and local independence be satisfied. In practice, these assumptions are often violated, which can lead to less accurate analyses and reduced validity of the results. Deep-Rasch, which integrates deep learning with Rasch modeling, has been proposed as an alternative measurement framework to overcome these limitations. This study examines the potential of Deep-Rasch as an alternative to Rasch modeling using student response data from 17 final semester examinations at Universitas Terbuka (UT), with sample sizes ranging from 33 to 11,504 students. Most examinations consisted of 30 multiple-choice items. The analyses showed that several datasets violated one or both assumptions of Rasch modeling. Nevertheless, Deep-Rasch performed comparably to conventional Rasch modeling in estimating item difficulty and student ability parameters, as well as in predicting student responses. Remarkably, for the smallest sample size (\emph{n} = 33), Deep-Rasch exhibited slightly better performance than Rasch modeling.