Real-Time Webcam-Based Hand Gesture Recognition with Face Authentication for 3D Drone Simulation in Godot Engine - Muhammad Rizqi Sholahuddin; Siti Dwi Setiarini; Ardhian Ekawijana; Muhammad Samudera; Firas Atqiya (Media Jurnal Informatika, 2026)
Most hand gesture control systems for drones depend on specialized hardware such as Leap Motion or Kinect, which raises the cost barrier for educational institutions in developing countries. Integrating face authentication within the same low-cost pipeline remains under-explored. This study develops a real-time, webcam-based system that combines Google MediaPipe hand tracking with facial authentication and a Godot Engine 4.3 3D drone simulation for authenticated, responsive gesture control. A finger-counting algorithm classifies eight gestures across two hands. The left hand drives horizontal motion (forward, backward, left, right) and the right hand drives altitude and yaw (up, down, rotate left, rotate right). Commands travel over UDP to Godot, where a receiver node translates each packet into a native input action. Face authentication uses dlib and the face_recognition library with a 60-frame login counter. All metrics were collected under a fixed condition (normal lighting 300–500 lux, 0.8 m, one subject). The system achieved 100% gesture accuracy across 160 trials, 35.6 FPS pipeline throughput, 0.33 ms one-way UDP latency with 0% packet loss, and 23.9 ms end-to-end gesture-to-drone latency. Face authentication scored 100% recognition with 0% FRR and 19.0% FAR against an unregistered face at the default 0.6 tolerance. A standard-webcam pipeline built entirely from open-source components can deliver responsive, authenticated gesture control for interactive drone simulation, though the single-subject evaluation is an upper bound requiring multi-subject validation. However, the 100% accuracy represents an upper bound as evaluation was limited to a single subject under controlled lighting (300–500 lux) and a fixed distance (0.8 m), requiring further validation across diverse users and environments