Claim Missing Document
Check
Articles

Found 1 Documents
Search
Journal : Jurnal Ilmu Komputer dan Informasi

Temporal Action Segmentation in Sign Language System for Bahasa Indonesia (SIBI) Videos Using Optical Flow-Based Approach I Dewa Made Bayu Atmaja Darmawan; Linawati; Sukadarmika, Gede; Wirastuti, Ni Made Ary Esta Dewi; Pulungan, Reza
Jurnal Ilmu Komputer dan Informasi Vol. 17 No. 2 (2024): Jurnal Ilmu Komputer dan Informasi (Journal of Computer Science and Informatio
Publisher : Faculty of Computer Science - Universitas Indonesia

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.21609/jiki.v17i2.1284

Abstract

Sign language (SL) is vital in fostering communication for the deaf and hard-of-hearing communities. Continuous Sign Language Translation (CSLT) is a work that translates sign language into spoken language. CSLT translation is done by changing continuous forms into isolated signs. Segmenting morpheme signs from phrase signs has several challenges, such as the availability of annotated datasets and the complexity of continuous gesture movements. The Indonesian Sign Language (SIBI) system follows Indonesian grammatical norms, including word formation, in contrast to other sign languages with rules derived from their spoken language. In SIBI, a word can consist of a root word and an affix word. Therefore, temporal action segmentation in SIBI is important to reconstruct the results of translating each sign into spoken Indonesian sentences. This research uses an optical flow approach to segment temporal actions in SIBI videos. Optical flow methods that calculate changes in intensity between adjacent frames can be used to determine the occurrence of sign movement or vice versa to determine the delay between sign movements. The absence of intensity differences between the two frames indicates the boundary between sign gestures. This study tested the use of dense optical flow on videos containing SIBI sentences taken from 3 signers. Evaluation is done on several parameters in the dense optical flow algorithm, such as threshold size, PyrScale, and WinSize, to obtain the best accuracy. This paper shows that the optical flow algorithm successfully performs segmentation, as measured by Perf and F1r. The experimental results showed that the highest Perf and F1r yields were 0.8298 and 0.8524, respectively.
Co-Authors Adinda Putri Ambarwati Afini Fikriyani agustina, karina enny Amelia Putri Cahyanti Andriyanto, Nazwa Ayumi Andy Kurniawan Andy Kurniawan, Andy Ariyanti, Maezora Putri Ayu Alfiani Azharul M, Muh Rizal Badrus Zaman Barep Setiaji Bawazir, Fatmah Cholid Budiastra, IN Daniel Happy Putra Delima, Anisa Dessy Hariyanti, NK Drajad Aprilianto Nugroho Erliana Evi Rosalina Widyayanti Fadhillah, Revita Rizki Farida Nur Mudmainah Wahyuningsih Ferdian Utama Gede Sukadarmika I Dewa Made Bayu Atmaja Darmawan I Made Oka Widyantara I Nyoman Gunantara I Putu Agus Eka Pratama I Putu Gede Hendra Suputra Insiatiningsih Irhamudin juniati Juniati, Ni Ketut Khoirul Anam Kurnia Gusti, Yenni Kurniawan , Andy Larasati, Anggia Bintang Linda Suciati Lukmanul Hakim Maisura Marbun, Rifka Angelina Wati Mareta Selvia Riswahyuning MHD. Reza M.I. Pulungan Mohammad Irvan Bayu Saputra Muh. Rizal Muhammad Fatkhul Ikhwansyah Naufal Friza Asraful Arifin Ngurah Indra ER Ni Made Ary Esta Dewi Wirastuti Nirawati Pribadi Nur Rohman Hidayat Nurdiwaty , Diah Nursolikin Nyoman Putra Sastra Pertiwi, Santhi Rachmawati, Atika Nisaa Ramadhan, Aqila Shabrina Dwi Randi, Muhammad Rasyidah Rawati Wawuru Rimas Sadha Abdi Putra Riski Masita Rizkiyani, Ika Rosalina Widyayanti, Evy Sabitillah, Lisa Sagung Putri M. E. Purwani Sari, Yulita Fatma Sofia Widiana Sofianiyatin, Lilik Sofiati Sofiati Sofiati, Sofiati Suhardi Sukadarmika, I Gede Sutrischastini, Ary Suwarni Suwarni Wahyu Kukuh Prayoga Widiawati, Hestin Sri Wulan Wulan Feronika Maharani Yenni Kurnia Gusti Yoga, I Putu Sudharma