Claim Missing Document
Check
Articles

Found 13 Documents
Search

Comparison of methods for detecting anomalous behavior on large-scale computer-based exams based on response time and responses Deni Hadiana; Bahrul Hayat; Burhanuddin Tola
REID (Research and Evaluation in Education) Vol 6, No 2 (2020)
Publisher : Program Pascasarjana Universitas Negeri Yogyakarta & HEPI

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.21831/reid.v6i2.31260

Abstract

This study aims to determine the anomalous index (indeks anomali or IA) that considers both response time and responses and compares it with response time effort (RTE) or rapid guessing (tebakan cepat or TC) on various thresholds. Response time and responses from 732 examinees are in natural science subjects consist of 40 multiple choice items with four answer choices. Response time and responses are analyzed to obtain descriptive statistics related to them, calculate the TC and IA index using two methods of the threshold, the first method (M1) is a visualization of identification, and the second method (M2) is based on the amount of time spent responding to each item related to the complexity of items, as proposed by Nitko. The performance of the IA and TC scores is compared related to validity and reliability. The coefficient alpha of IAM1 score 0.84, the coefficient alpha of IAM2 0.82. Both values of the alpha coefficient have fulfilled the reliability requirements of the index determination. The IA proposed in this study has a high correlation with ERP, which is commonly used to determine the solution behavior's magnitude and rapid guessing. The correlation value of IAM1 with TCM1 0.86, the correlation value of IAM2 with TCM2 0.89, and this high correlation value shows that there is a strong relationship between IA and TC. Determination of threshold time uses three categories of multiple choices item that reveal IA and TC distributions that are close to normal distribution so that it reflects natural empirical conditions.
Metacognitive Skills Assessment in Research-Proposal Writing (MSARPW) in the Indonesian University Context: Scale Development and Validation Using Multidimensional Item Response Models Bahrul Hayat; Wardani Rahayu; Muhammad Dwirifqi Kharisma Putra; Iva Sarifah; Tina Deviana; Valendra Granitha Shandika Puri; Khairunesa Isa
JP3I (Jurnal Pengukuran Psikologi dan Pendidikan Indonesia) Vol 12, No 1 (2023): JP3I
Publisher : Fakultas Psikologi UIN Syarif Hidayatullah Jakarta

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.15408/jp3i.v12i1.31679

Abstract

This study aimed to develop the Metacognitive Skills Assessment in a Research Proposal Writing Context (MSARPW), a multidimensional measure, based on student perspectives, for assessing metacognitive skills in thesis writing, and assess the instrument’s psychometric properties using item factor analysis (IFA) and multidimensional item response models. The 40-item MSARPW was administered to 602 Indonesian university students (Mage = 25.254 SDage = 6.854). The IFA showed that the two-dimensional factor structure of MSARPW was satisfactory; however, only 24 of the 40 items were found to fit the model. Multidimensional graded response models (MGRM) were applied to the subsequent 24-item MSARPW, which showed that one item (Item 4) did not satisfy the criteria. The estimated reliabilities of each subscale showed that the 23-item MSARPW has good internal consistency (0.891 and 0.902). To conclude, the 23-item MSARPW appears to be a valid and reliable tool for assessing metacognitive skills in the context of research-proposal writing among Indonesian university students.
Metacognitive Skills Assessment in Research-Proposal Writing (MSARPW) in the Indonesian University Context: Scale Development and Validation Using Multidimensional Item Response Models Bahrul Hayat; Wardani Rahayu; Muhammad Dwirifqi Kharisma Putra; Iva Sarifah; Tina Deviana; Valendra Granitha Shandika Puri; Khairunesa Isa
JP3I (Jurnal Pengukuran Psikologi dan Pendidikan Indonesia) Vol 12, No 1 (2023): JP3I
Publisher : Fakultas Psikologi UIN Syarif Hidayatullah Jakarta

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.15408/jp3i.v12i1.31679

Abstract

This study aimed to develop the Metacognitive Skills Assessment in a Research Proposal Writing Context (MSARPW), a multidimensional measure, based on student perspectives, for assessing metacognitive skills in thesis writing, and assess the instrument’s psychometric properties using item factor analysis (IFA) and multidimensional item response models. The 40-item MSARPW was administered to 602 Indonesian university students (Mage = 25.254 SDage = 6.854). The IFA showed that the two-dimensional factor structure of MSARPW was satisfactory; however, only 24 of the 40 items were found to fit the model. Multidimensional graded response models (MGRM) were applied to the subsequent 24-item MSARPW, which showed that one item (Item 4) did not satisfy the criteria. The estimated reliabilities of each subscale showed that the 23-item MSARPW has good internal consistency (0.891 and 0.902). To conclude, the 23-item MSARPW appears to be a valid and reliable tool for assessing metacognitive skills in the context of research-proposal writing among Indonesian university students.