Bayu Sulistiyanto Ipung Sutejo
Universitas Esa Unggul, Jakarta, Indonesia

Published : 2 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 2 Documents
Search

Development and Usability Evaluation of an AI-Based Vocal Emotion Assessment Prototype for Vocational Communication Training Bayu Sulistiyanto Ipung Sutejo; Muhammad Revin Putra Fahlefi; Habibullah Akbar; Andriyanti Asianto; Diah Aryani; Ega Shela Marsiani
Chatra: Jurnal Pendidikan dan Pengajaran Vol. 4 No. 3 (2026): Chatra: Jurnal Pendidikan dan Pengajaran (In Press)
Publisher : CV. Yazri Aksara Nusantara

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.62238/chatra.v4i3.476

Abstract

Vocational communication training requires opportunities for learners to practise vocal delivery and receive timely feedback, yet conventional assessment often depends on direct instructor observation. This study developed and evaluated a web-based AI-assisted vocal emotion assessment prototype for vocational communication practice. Using evolutionary prototyping, the system integrated browser-based audio recording and file upload, signal preprocessing, emotion2vec+ inference, probability visualization, and automated feedback. Functional performance was examined through eight black-box testing scenarios, while perceived usability was evaluated using the System Usability Scale with 15 participants comprising 12 vocational students and three instructors. All eight functional scenarios operated successfully, with client-server response times of approximately 0.45–0.65 seconds. The prototype achieved a SUS score of 76.98, indicating favorable perceived usability, particularly for ease of use and learnability. These findings support the prototype's preliminary feasibility as a formative practice tool, although classification validity and educational effectiveness require further evaluation using larger samples, expert-validated emotion labels, and authentic vocational communication settings.Pelatihan komunikasi vokasional memerlukan kesempatan bagi peserta didik untuk mempraktikkan penyampaian vokal dan memperoleh umpan balik secara tepat waktu, sedangkan asesmen konvensional masih banyak bergantung pada pengamatan langsung instruktur. Penelitian ini mengembangkan dan mengevaluasi prototipe asesmen emosi vokal berbasis web dengan dukungan kecerdasan buatan untuk latihan komunikasi vokasional. Pengembangan menggunakan evolutionary prototyping dengan mengintegrasikan perekaman dan pengunggahan audio, prapemrosesan sinyal, inferensi emotion2vec+, visualisasi probabilitas, serta umpan balik otomatis. Kinerja fungsional diuji melalui delapan skenario black-box testing, sedangkan persepsi usability dievaluasi menggunakan System Usability Scale terhadap 15 partisipan yang terdiri atas 12 peserta didik vokasi dan tiga instruktur. Seluruh skenario pengujian fungsional berhasil dijalankan dengan waktu respons client-server sekitar 0,45–0,65 detik. Prototipe memperoleh skor SUS sebesar 76,98 yang menunjukkan persepsi usability yang positif, terutama pada kemudahan penggunaan dan pembelajaran sistem. Temuan ini mendukung kelayakan awal prototipe sebagai alat latihan formatif, tetapi validitas klasifikasi dan efektivitas pendidikan masih memerlukan pengujian lebih lanjut dengan sampel yang lebih luas, label emosi tervalidasi pakar, dan konteks komunikasi vokasional autentik.
Evaluating the Usability of an RF–LSTM Learning Platform with Explainable AI among University Students Diah Aryani; Jefry Sunupurwa Asri; Habibullah Akbar; Muhammad Fikri; Bayu Sulistiyanto Ipung Sutejo
Chatra: Jurnal Pendidikan dan Pengajaran Vol. 4 No. 3 (2026): Chatra: Jurnal Pendidikan dan Pengajaran (In Press)
Publisher : CV. Yazri Aksara Nusantara

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.62238/chatra.v4i3.491

Abstract

Educational AI platforms require usable interfaces that enable students to interpret complex predictive and explainability outputs. This study evaluated the usability of an RF-LSTM learning platform incorporating explainable artificial intelligence among university students. A multidimensional usability evaluation involved 40 Informatics Engineering and Computer Science students who completed six structured tasks covering data exploration, preprocessing, model prediction, performance comparison, SHAP interpretation, and navigation. Usability evidence was obtained from task performance, structured observations, the System Usability Scale (SUS), and open-ended feedback. Task completion ranged from 85% to 100%, while the mean SUS score of 76.7 indicated positive overall usability. Greater difficulties emerged when students compared performance metrics and interpreted SHAP visualizations, particularly regarding technical terminology, visual presentation, navigation, and system feedback. The findings suggest that algorithmic explainability does not automatically ensure comprehensibility during student interaction. Contextual explanations, clearer navigation cues, scalable SHAP visualizations, and processing feedback emerged as design priorities, although the findings remain limited to students within a higher education context.Platform pendidikan berbasis AI memerlukan antarmuka yang mudah digunakan agar mahasiswa mampu menginterpretasikan keluaran prediktif dan explainability yang kompleks. Penelitian ini mengevaluasi usabilitas platform pembelajaran RF-LSTM yang mengintegrasikan explainable artificial intelligence pada mahasiswa. Evaluasi usabilitas multidimensional melibatkan 40 mahasiswa Teknik Informatika dan Ilmu Komputer yang menyelesaikan enam tugas terstruktur meliputi eksplorasi data, preprocessing, prediksi model, perbandingan performa, interpretasi SHAP, dan navigasi. Bukti usabilitas diperoleh melalui performa tugas, observasi terstruktur, System Usability Scale (SUS), dan umpan balik terbuka. Tingkat penyelesaian tugas berkisar antara 85% hingga 100%, sedangkan rerata skor SUS sebesar 76,7 menunjukkan usabilitas keseluruhan yang positif. Kesulitan lebih terlihat ketika mahasiswa membandingkan metrik performa dan menginterpretasikan visualisasi SHAP, terutama terkait terminologi teknis, penyajian visual, navigasi, dan umpan balik sistem. Temuan menunjukkan bahwa explainability algoritmik tidak secara otomatis menjamin keterpahaman dalam interaksi mahasiswa. Penjelasan kontekstual, penanda navigasi yang lebih jelas, visualisasi SHAP yang dapat disesuaikan, dan umpan balik pemrosesan menjadi prioritas pengembangan, meskipun temuan terbatas pada mahasiswa dalam konteks perguruan tinggi.