Article Per Year (5 Year)
p-Index From 2020 - 2025
1.259
P-Index
This Author published in this journals
All Journal Jurnal Penelitian dan Evaluasi Pendidikan Ars: Jurnal Seni Rupa dan Desain Journal of Visual Art and Design MUDRA Jurnal Seni Budaya JMM (Jurnal Masyarakat Mandiri) Journal of Educational Research and Evaluation JISAE (Journal of Indonesian Student Assesment and Evaluation)
Putu Diah Ari Kusumadewi
Universitas Negeri Yogyakarta
Religion Arts Humanities Decision Sciences, Operations Research & Management Education Languange, Linguistic, Communication & Media Public Health Social Sciences Other

Published : 7 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search
Journal : Journal of Educational Research and Evaluation

 1 
Assessing the Reliability of Performance-Based Evaluation in Fashion Education: A Generalizability Theory Approach Kusumadewi, Putu Diah Ari; Rosnawati, Raden
Journal of Education Reseach and Evaluation Vol 9 No 3 (2025): August
Publisher : LPPM Undiksha

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.23887/jere.v9i3.100248

Abstract

This study addresses the issue of score inconsistency in performance-based assessment (PBA), particularly in creative disciplines where fairness and reliability are crucial. The main objectives of this study were to evaluate the reliability of PBA scores and analyze sources of measurement error using Generalizability Theory (G-Theory). This quantitative study employed a fully crossed (p × r × d) design in an eveningwear construction course. A total of 70 student fashion products were assessed by four assessors consisting of two internal lecturers and two external industry professionals byusing a five-dimensional rubric. Data were collected through standardized scoring sheets and analyzed using linear mixed-effects modeling in R software (lme4 package). Variance components were estimated in the Generalizability Study (G-Study), followed by simulations in the Decision Study (D-Study) to test various assessment configurations. The G-Study results indicated that the largest variance component came from residual error (50.42%), followed by product differences (43.44%), while the contributions of assessors and dimensions were relatively small. The D-Study showed that increasing the number of raters and rubric dimensions significantly increased reliability; a configuration of four raters and five dimensions resulted in a generalizability coefficient of 0.9185. These findings confirm that G-Theory is effective for diagnosing and optimizing reliability in the context of creative assessment. This research provides practical insights for educators in designing fairer and more reliable assessment systems, and opens up opportunities for the application of psychometrics in vocational and design education.
Co-Authors Emy Budiastuti Mohammad Adam Jerusalem NUR KHOLIFAH Raden Rosnawati Sri Wening Sudji Munadi Widihastuti