Claim Missing Document
Check
Articles

Psychometric Validation of an AI-Based Evaluation System for Identifying Discrepancies in Learning Processes P. Wayan Arta Suyasa; I Gusti Ngurah Pujawan; Dewa Gede Hendra Divayana; I Dewa Ayu Made Budhyani; I Made Sugiarta; I Made Candiasa
Journal of Applied Data Sciences Vol 7, No 2: May 2026
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v7i2.1168

Abstract

This research advances the field of educational evaluation by designing and psychometrically validating an artificial intelligence (AI)- based diagnostic tool to detect discrepancies in university learning processes. The main novelty is the integration of the Provus Discrepancy Model combined with a forward-chaining inference engine. This research aims to transform evaluation from an administrative activity to an ongoing process of improvement. The tool was developed and validated through a sequential mixed-methods approach with 400 participants from 3 state universities and 8 evaluation experts. Results from the study provide evidence that the validated system created a substantial range of psychometric characteristics. These psychometric characteristics include strong content validity (SD-CVI/Ave = 0.94); high internal consistency and reliability (Cronbach's α = 0.94); solid construct validity as demonstrated through Confirmatory Factor Analysis (CFA) (CFI = 0.94; RMSEA = 0.054) and a substantial range of predictive analytics (diagnostic learning analytics), which the AI learning analytics engine evaluated learning discrepancies with a 92.4% diagnostic accuracy (47.4% more accurate than manual evaluation methods). The system's validated usefulness is demonstrated through high system usability (SUS = 88.2); high practical utility (85% total score on the Pragmatic Utility Assessment); significant utility (real-world) practical utility (detected 45 discrepancy patterns), cost efficiency (73% cost and 67% analysis time compared to traditional methods), and a range of analytics (predictive and learning discrepancy analytics). The significant contribution of this study is the development of the world's first integrated AI evaluation system that meets high methodological and psychometric standards, along with a set of real-time diagnostic analytics. Ultimately, this study developed the first truly integrated, novel paradigm evaluation system that combined the historically established evaluation construct and mechanisms with the most advanced AI capabilities, providing educators and institutions with evaluation tools to deliver data-driven pedagogical strategies and interventions in higher education. 
Transformasi Asesmen Matematika pada Era Kecerdasan Buatan: Sebuah Systematic Literature Review tentang Praktik Asesmen, Umpan Balik, Learning Analytics, dan Arah Penelitian Masa Depan Lisda Ramdhani; I Gusti Lanang Agung Parwata; I Gede Ratnaya; I Dewa Ayu Made Budhyani
Mandalika Mathematics and Educations Journal Vol 8 No 3 (2026): Edisi September
Publisher : FKIP Universitas Mataram

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29303/jm.v8i3.12725

Abstract

The rapid advancement of Artificial Intelligence (AI) has significantly transformed mathematics assessment by enabling adaptive assessment, automated feedback, learning analytics, and data-driven instructional decision-making. This study aimed to synthesize current evidence regarding AI-based mathematics assessment, feedback practices, learning analytics, and future research directions. A Systematic Literature Review (SLR) was conducted following the PRISMA 2020 guidelines using the Scopus database. The search covered publications from 2015 to 2026 with the query related to mathematics education, generative AI, artificial intelligence, assessment, and feedback. After applying inclusion and exclusion criteria, 33 articles were selected for analysis. The findings indicate that AI has shifted mathematics assessment from traditional summative evaluation toward adaptive, formative, and personalized assessment. AI-generated feedback and learning analytics improve students' problem-solving skills, conceptual understanding, and teachers' instructional decisions. Nevertheless, issues concerning validity, ethical use, data privacy, and teachers' AI literacy remain major challenges. Future studies should develop ethical, explainable, and human-centered AI assessment frameworks that strengthen mathematics learning while preserving pedagogical quality.
EVALUASI DISTRIBUSI LEVEL KOGNITIF DAN DIMENSI PENGETAHUAN SOAL PENDIDIKAN AGAMA HINDU DAN BUDI PEKERTI BERDASARKAN TAKSONOMI BLOOM REVISI Rudiarta, I Wayan; Budhyani, I Dewa Ayu Made; Ratnaya, I Gede; Parwata, I Gusti Lanang Agung
Guna Widya: Jurnal Pendidikan Hindu Vol 13 No 2 (2026): Jurnal Guna Widya Volume 13 Nomor 2 September 2026
Publisher : Universitas Hindu Negeri I Gusti Bagus Sugriwa Denpasar

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.25078/gw.v13i2.6647

Abstract

This study aims to analyze the distribution of cognitive levels and knowledge dimensions inthe Final Semester Summative Assessment of Hindu Religious Education and CharacterEducation for Grades III, IV, and V of elementary school based on Anderson andKrathwohl’s revised Bloom’s Taxonomy. The study employed a quantitative descriptiveapproach using document analysis as the research method. The data source consisted ofHindu Religious Education and Character Education Final Semester SummativeAssessment items, which were analyzed according to cognitive levels (C1–C6) and thedimensions of factual, conceptual, procedural, and metacognitive knowledge. Data werecollected through documentation and item identification using indicators derived from therevised Bloom’s Taxonomy. The data analysis procedure involved classifying the assessmentitems based on cognitive levels and knowledge dimensions, followed by calculating theirfrequencies and percentages. The findings revealed that the assessment items werepredominantly categorized as Lower Order Thinking Skills (LOTS), particularly at theremembering (C1) and understanding (C2) levels. Conceptual knowledge emerged as themost dominant knowledge dimension, while procedural and metacognitive dimensionsremained relatively limited. Furthermore, no assessment items were identified at the applyinglevel (C3) across all grade levels. These findings indicate that the evaluation practices inHindu Religious Education and Character Education learning remain largely orientedtoward conceptual mastery and have not fully accommodated the development of students’critical, reflective, and applicative thinking skills. Therefore, there is a need to develop moreintegrative and HOTS-oriented assessment instruments
THE ANALYSIS OF AUTHENTIC ASSESSMENT FOR YOUNG CHILDREN DEVELOPMENT Md. Yudyantara Risadi; I Gusti Lanang Agung Parwata; I Gede Ratnaya; I Dewa Ayu Made Budhyani
PRATAMA WIDYA : Jurnal Pendidikan Anak Usia Dini Vol 11 No 2 (2026): Pratama Widya Oktober 2026
Publisher : Universitas Hindu Negeri I Gusti Bagus Sugriwa Denpasar

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.25078/pw.v11i2.6855

Abstract

This study explores the implementation of authentic assessment in monitoring young children’s development in a kindergarten setting in Denpasar, Bali. Authentic assessment is increasingly recognized as a more suitable approach than traditional testing because it evaluates children in real-life contexts and captures multiple developmental domains. Using a qualitative case study design, data were collected through interviews, classroom observations, and document analysis involving one principal and four teachers. The findings highlight four key practices: daily observation and documentation, portfolio-based assessment with a star rating rubric, structured feedback and critical questioning, and consideration of children’s emotional states. Daily records and checklists provided systematic evidence of developmental progress, while portfolios and star ratings offered insights into independence and creativity. Feedback cycles and critical questioning supported deeper learning and parent-teacher collaboration. Importantly, children’s mood and character were found to significantly influence assessment outcomes, requiring flexible and child-centered negotiation strategies. Overall, authentic assessment functioned not only as an evaluation tool but also as a holistic approach to support children’s development. The study contributes to strengthening authentic assessment practices in early childhood education by emphasizing continuous observation, responsive teaching, and collaborative communication with parents.
Co-Authors ., Anak Agung Ratih Urmila Santhi ., Dewa Ayu Dwitya p ., Dewa Ayu Widia Natalia ., Kadek Monik Widi Hutami ., Khaeriyah Ramdani ., Linda Lestari ., Luh Putu Sri Okta Pradinda ., Ni Kadek Asri Cahyani ., Ni Kadek Emma Pratiwi ., Ni Ketut Dewi Tara Dipa ., Ni Made Oktasari ., Ni Putu Ayu Ari Anggarwati ., Putu Ayu Laksmidevi ., Putu Gitta Fortuna ., Putu Nada Widiya Purnashita ., Sohriah abdillah, awaluddin muhammad Anak Agung Ratih Urmila Santhi . Anastasia Bita . Asti Noviana . awaluddin muhammad abdillah Baiq Rina Amalia Safitri Cahyani, Ni Putu Riska Dewa Ayu Dwitya p . Dewa Ayu Putu Widya Wulandari . Dewa Ayu Widia Natalia . Dewa Ayu Widia Natalia Natalia Dewa Gede Hendra Divayana, Dewa Gede Hendra Dewa Nyoman Oka Dika Ayu Fitriana Dr. I Made Sugiarta, M.Si. . Fitria . Hairil Wadi Hutami, K.M. Widi I Gede Ratnaya I Gede Sudirtha I Gst Lanang Agung Parwata I Gusti Ayu Putri Candra Dewi . I Gusti Ngurah Pujawan I Gusti Ngurah Pujawan I Kadek Edi Yudiana I Made Candiasa I Wayan Rudiarta I.G. Sudirtha Ida Ayu Reviena Damasanti Juniawati, Ni Kadek Windi K.M. Widi Hutami Kadek Monik Widi Hutami . Karisnayani, Made Khaeriyah Ramdani . Komang Erna Astari . Linda Lestari . Lisda Ramdhani Luh Putu Sri Okta Pradinda . Made Diah Angendari Made Sutajaya Maria Ellysabeth Waha Mare . Maylina Sari, Putri Md. Yudyantara Risadi Mujiburrahman Mujiburrahman Natalia, Dewa Ayu Widia Natalia NENI PEBRIYANA . Ni Kadek Asri Cahyani . Ni Kadek Emma Pratiwi . Ni Kadek Windi Juniawati Ni Kadek Windi Juniawati . Ni Ketut Dewi Tara Dipa . Ni Ketut Widiartini Ni Made Oktasari Ni Made Oktasari . Ni Made Serma Wati Ni Nyoman Ratih Diana Utari Ni Nyoman Serma Adi Ni Putu Ayu Ari Anggarwati . Ni Putu Dewi Sri w . Ni Putu Wulan Hartawati Nyoman Eravitaliani Nyoman Maesha Bramanda Oktasari, Ni Made P. WAYAN ARTA SUYASA Puger, I Gusti Ngurah Putu Agus Mayuni Putu Ayu Laksmidevi . Putu Gitta Fortuna . Putu Kerti Nitiasih Putu Nada Widiya Purnashita . Ruhama'Ul Magfirah . Salma Hasna Hamiydah Sang Ayu Alit Wilantari . Sohriah . Sri Haerina Lailatul Janah Sudirtha, I.G. Utari, Ni Nyoman Ratih Diana Wadi, Hairil Widad Azizah Widiastini, Ni Wayan Eka Widyawati, Ni Komang Yuli Harniati .