Psychometric Evaluation of a START Triage Psychomotor Observation Instrument: Content Validity and Interrater Reliability - Irnu, Julfilkar Adnan; Faizatiwahida, Novia; Fadilah, Superzeki Zaidatul; Listyaningrum, Desy (KHD Production, 2026)
Accurate psychomotor assessment in Simple Triage and Rapid Treatment (START) is critical during mass-casualty incidents. Observation-based evaluations often yield inconsistent judgments when utilizing unvalidated instruments, compromising competency assessments. This study aimed to establish the content validity and interrater reliability of a newly developed psychomotor observation sheet for START triage among healthcare professionals. This methodological study evaluated a 12-item dichotomous observation sheet derived from standard START protocols. Content validity was determined by three emergency and disaster nursing experts using a four-point relevance scale, analyzed with Aiken's V. Interrater reliability was assessed with 36 medical staff members, independently evaluated by two trained raters. Agreement metrics included percent agreement and Cohen's kappa. Expert evaluation yielded a perfect Aiken's V of 1.00 across all 12 items, indicating exceptional content validity. Interrater reliability testing demonstrated percent agreement ranging from 94.44% to 100%, alongside Cohen's kappa coefficients between 0.85 and 1.00. These metrics significantly exceeded minimum acceptability thresholds, reflecting outstanding consistency and minimal scoring ambiguity among raters. The START triage psychomotor observation sheet exhibits robust content validity and excellent interrater reliability. This validated instrument provides educators with a standardized, objective tool for evaluating clinical competencies in disaster preparedness training, though future validation in high-fidelity simulation environments is recommended.