IJCCS (Indonesian Journal of Computing and Cybernetics Systems)
Vol 17, No 3 (2023): July

Error Action Recognition on Playing The Erhu Musical Instrument Using Hybrid Classification Method with 3D-CNN and LSTM

Aditya Permana (Universitas Gadjah Mada)
Timothy K. Shih (National Central University)
Aina Musdholifah (Universitas Gadjah Mada)
Anny Kartika Sari (Universitas Gadjah Mada)



Article Info

Publish Date
31 Jul 2023

Abstract

Erhu is a stringed instrument originating from China. In playing this instrument, there are rules on how to position the player's body and hold the instrument correctly. Therefore, a system is needed that can detect every movement of the Erhu player. This study will discuss action recognition on video using the 3DCNN and LSTM methods. The 3D Convolutional Neural Network method is a method that has a CNN base. To improve the ability to capture every information stored in every movement, combining an LSTM layer in the 3D-CNN model is necessary. LSTM is capable of handling the vanishing gradient problem faced by RNN. This research uses RGB video as a dataset, and there are three main parts in preprocessing and feature extraction. The three main parts are the body, erhu pole, and bow. To perform preprocessing and feature extraction, this study uses a body landmark to perform preprocessing and feature extraction on the body segment. In contrast, the erhu and bow segments use the Hough Lines algorithm. Furthermore, for the classification process, we propose two algorithms, namely, traditional algorithm and deep learning algorithm. These two-classification algorithms will produce an error message output from every movement of the erhu player.

Copyrights © 2023






Journal Info

Abbrev

ijccs

Publisher

Subject

Computer Science & IT Control & Systems Engineering

Description

Indonesian Journal of Computing and Cybernetics Systems (IJCCS), a two times annually provides a forum for the full range of scholarly study . IJCCS focuses on advanced computational intelligence, including the synergetic integration of neural networks, fuzzy logic and eveolutionary computation, so ...