Article Per Year (5 Year)
p-Index From 2021 - 2026
3.533
P-Index
This Author published in this journals
All Journal International Journal of Evaluation and Research in Education (IJERE) Jurnal BOSAPARIS: Pendidikan Kesejahteraan Keluarga Jurnal Pendidikan Teknologi dan Kejuruan WIDYA LAKSANA Mimbar Ilmu Jurnal Ilmiah Profesi Pendidikan Jurnal Paedagogy Eduproxima : Jurnal Ilmiah Pendidikan IPA Didaktika: Jurnal Kependidikan Journal of Applied Data Sciences Jurnal Ilmiah Ilmu Terapan Universitas Jambi Journal of Innovative Technology and Sustainability Education
I Dewa Ayu Made Budhyani
Universitas Pendidikan Ganesha
Biochemistry, Genetics & Molecular Biology Chemistry Computer Science & IT Control & Systems Engineering Decision Sciences, Operations Research & Management Earth & Planetary Sciences Education Engineering Industrial & Manufacturing Engineering Languange, Linguistic, Communication & Media Mathematics Physics Social Sciences Other

Published : 79 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search
Journal : Journal of Applied Data Sciences

 1 
Psychometric Validation of an AI-Based Evaluation System for Identifying Discrepancies in Learning Processes Suyasa, P. Wayan Arta; Pujawan, I Gusti Ngurah; Divayana, Dewa Gede Hendra; Budhyani, I Dewa Ayu Made; Sugiarta, I Made; Candiasa, I Made
Journal of Applied Data Sciences Vol 7, No 2: May 2026
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v7i2.1168

Abstract

This research advances the field of educational evaluation by designing and psychometrically validating an artificial intelligence (AI)- based diagnostic tool to detect discrepancies in university learning processes. The main novelty is the integration of the Provus Discrepancy Model combined with a forward-chaining inference engine. This research aims to transform evaluation from an administrative activity to an ongoing process of improvement. The tool was developed and validated through a sequential mixed-methods approach with 400 participants from 3 state universities and 8 evaluation experts. Results from the study provide evidence that the validated system created a substantial range of psychometric characteristics. These psychometric characteristics include strong content validity (SD-CVI/Ave = 0.94); high internal consistency and reliability (Cronbach's α = 0.94); solid construct validity as demonstrated through Confirmatory Factor Analysis (CFA) (CFI = 0.94; RMSEA = 0.054) and a substantial range of predictive analytics (diagnostic learning analytics), which the AI learning analytics engine evaluated learning discrepancies with a 92.4% diagnostic accuracy (47.4% more accurate than manual evaluation methods). The system's validated usefulness is demonstrated through high system usability (SUS = 88.2); high practical utility (85% total score on the Pragmatic Utility Assessment); significant utility (real-world) practical utility (detected 45 discrepancy patterns), cost efficiency (73% cost and 67% analysis time compared to traditional methods), and a range of analytics (predictive and learning discrepancy analytics). The significant contribution of this study is the development of the world's first integrated AI evaluation system that meets high methodological and psychometric standards, along with a set of real-time diagnostic analytics. Ultimately, this study developed the first truly integrated, novel paradigm evaluation system that combined the historically established evaluation construct and mechanisms with the most advanced AI capabilities, providing educators and institutions with evaluation tools to deliver data-driven pedagogical strategies and interventions in higher education. 
Co-Authors ., Anak Agung Ratih Urmila Santhi ., Dewa Ayu Dwitya p ., Dewa Ayu Widia Natalia ., Kadek Monik Widi Hutami ., Khaeriyah Ramdani ., Linda Lestari ., Luh Putu Sri Okta Pradinda ., Ni Kadek Asri Cahyani ., Ni Kadek Emma Pratiwi ., Ni Ketut Dewi Tara Dipa ., Ni Made Oktasari ., Ni Putu Ayu Ari Anggarwati ., Putu Ayu Laksmidevi ., Putu Gitta Fortuna ., Putu Nada Widiya Purnashita ., Sohriah abdillah, awaluddin muhammad Anak Agung Ratih Urmila Santhi . Anastasia Bita . Asti Noviana . awaluddin muhammad abdillah Cahyani, Ni Putu Riska Dewa Ayu Dwitya p . Dewa Ayu Putu Widya Wulandari . Dewa Ayu Widia Natalia . Dewa Ayu Widia Natalia Natalia Dewa Gede Hendra Divayana, Dewa Gede Hendra Dika Ayu Fitriana Dr. I Made Sugiarta, M.Si. . Fitria . Habibi, Baiq Azmi Sukroyanti, Lovy Herayanti, Wirawan Putrayadi, dan Bq. Rina Amalia Safitri Hairil Wadi Hutami, K.M. Widi I Gede Ratnaya I Gede Sudirtha I Gst Lanang Agung Parwata I Gusti Ayu Putri Candra Dewi . I Gusti Ngurah Pujawan I Kadek Edi Yudiana I Komang Wisnu Budi Wijaya I Made Candiasa I.G. Sudirtha Ida Ayu Reviena Damasanti Juniawati, Ni Kadek Windi K.M. Widi Hutami Kadek Monik Widi Hutami . Karisnayani, Made Khaeriyah Ramdani . Komang Erna Astari . Linda Lestari . Luh Putu Sri Okta Pradinda . Made Diah Angendari Made Sutajaya Maria Ellysabeth Waha Mare . Maylina Sari, Putri Mujiburrahman Mujiburrahman Natalia, Dewa Ayu Widia Natalia NENI PEBRIYANA . Ni Kadek Asri Cahyani . Ni Kadek Emma Pratiwi . Ni Kadek Windi Juniawati Ni Kadek Windi Juniawati . Ni Ketut Dewi Tara Dipa . Ni Ketut Widiartini Ni Made Oktasari Ni Made Oktasari . Ni Nyoman Ratih Diana Utari Ni Putu Ayu Ari Anggarwati . Ni Putu Dewi Sri w . Ni Putu Wulan Hartawati Oktasari, Ni Made P. WAYAN ARTA SUYASA Puger, I Gusti Ngurah Putu Agus Mayuni Putu Ayu Laksmidevi . Putu Gitta Fortuna . Putu Kerti Nitiasih Putu Nada Widiya Purnashita . Ruhama'Ul Magfirah . Salma Hasna Hamiydah Sang Ayu Alit Wilantari . Sohriah . Sri Haerina Lailatul Janah Sudirtha, I.G. Utari, Ni Nyoman Ratih Diana Wadi, Hairil Widad Azizah Widiastini, Ni Wayan Eka Widyawati, Ni Komang Windi Ayu Lestari Yuli Harniati .