Claim Missing Document
Check
Articles

Found 1 Documents
Search

PENGEMBANGAN GAME EDUKASI AR DENGAN NPC GURU VIRTUAL BERBASIS MEDIAPIPE DAN AI VOICE INTERACTION Muh. Hasrul; Fahrim Irhamna Rachman; Runal Rezkiawan
Nusantara Hasana Journal Vol. 6 No. 3 (2026): Nusantara Hasana Journal, August 2026
Publisher : Yayasan Nusantara Hasana Berdikari

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.59003/nhj.v6i3.2328

Abstract

The development of interactive learning media has been supported by the integration of Augmented Reality (AR), Artificial Intelligence (AI), and voice interaction technologies. However, learning applications that integrate AR with virtual teacher characters and multiple forms of interaction are still challenging to implement, particularly in terms of system integration and device limitations. This study aims to develop an educational game based on Augmented Reality with a Virtual Teacher NPC that integrates AI, voice interaction, ML Kit Optical Character Recognition (OCR), MediaPipe, chat, and quiz features. The application was developed using Unity and AR Foundation, with Google Gemini used to generate AI responses. Voice interaction was implemented through Speech-to-Text (STT) and Text-to-Speech (TTS), while ML Kit OCR was used to recognize text from scanned learning materials. MediaPipe was implemented to process visual information from the user's face as a supporting component for NPC interaction. The system was evaluated using Black Box Testing, System Usability Scale (SUS), and User Acceptance Testing (UAT). The results of Black Box Testing showed that most of the main functions operated according to the designed scenarios, although several functions were partially successful due to OCR image quality and MediaPipe implementation limitations. Testing with 20 respondents produced an average SUS score of 63.88, while UAT produced an average score of 3.58 out of 5 or 71.61%. These results indicate that the developed application can function as an interactive educational media, although improvements are still required in usability, OCR performance, and MediaPipe implementation.