<![CDATA[ Maintaining the quality of educational textbooks requires valid, reliable, and transparent evaluation.However, multi-rater assessments may be affected by differences in judgment and rater severity. Thisstudy examined a textbook evaluation system using the Many-Facet Rasch Model (MFRM). Sixevaluators assessed three books using a 12-item instrument. The findings revealed substantial variationin rater severity, including one unusually severe rater, indicating a need to improve consistency inevaluators’ judgments. Most items functioned adequately, but two showed misfit and required revisionor removal. The analysis also distinguished differences in the estimated quality of the three books whileaccounting for rater effects. These findings demonstrate the practical value of MFRM for strengtheningthe fairness and credibility of textbook evaluation. Institutions responsible for textbook assessment shouldregularly review evaluation instruments, clarify rating criteria, and provide training and calibration topromote consistent judgments. They should also investigate unusually severe or inconsistent ratingpatterns and revise or remove items that do not function as intended. Incorporating MFRM intoevaluation procedures can help institutions identify weaknesses in instruments and rating practices,support evidence-based textbook selection, and improve quality assurance. These steps can helppolicymakers establish more transparent and rigorous processes for evaluating educational materials. ]]>
Copyrights © 2026
