<![CDATA[Introduction: Angklung is a traditional Indonesian musical instrument that continues to evolve through digital technology. However, computer vision–based gesture recognition for controlling physical angklung instruments remains limited. This study investigates landmark-based recognition of Kodály hand signs and evaluates its application for real-time angklung interaction. Method: Hand landmarks were extracted using MediaPipe Hands from RGB camera input. Each gesture was represented by 63 normalized numerical features derived from 21 landmarks. The dataset consists of 8,000 images representing eight Kodály gesture classes (Do–Do'). Gesture classification was performed using the Extreme Gradient Boosting (XGBoost) algorithm. Model evaluation applied a subject-independent two-fold scheme using accuracy, precision, recall, F1-score, and confusion matrix analysis. Real-time system trials were conducted under different lighting conditions and capture distances, and TCP communication with an ESP32 controller was evaluated. Results: The model achieved 96.63% accuracy in Fold 1 and 96.40% in Fold 2. Misclassifications were mainly observed between visually similar gestures, particularly La and Mi. Separate real-time system trials showed consistent recognition under bright lighting, while accuracy decreased under dim lighting, especially for Do (90%) and Mi (86.7%). Gesture recognition remained reliable up to approximately 1.5 m. TCP testing over 200 command events recorded 0% failed acknowledgments with a mean round-trip time of 87.36 ms. Conclusion: These indicate that landmark-based Kodály gesture classification using MediaPipe Hands and XGBoost can support real-time angklung interaction under controlled conditions, although improvements are needed for low-light environments and visually similar gestures]]>
