Claim Missing Document
Check
Articles

Found 1 Documents
Search
Journal : JOIV : International Journal on Informatics Visualization

Multi Task Deep Learning with Transformer Encoder Decoder for Semantic Segmentation Indah, Komang Ayu Triana; Darma Putra, I Ketut Gede; Sudarma, Made; Hartati, Rukmi Sari
JOIV : International Journal on Informatics Visualization Vol 9, No 1 (2025)
Publisher : Society of Visual Informatics

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.62527/joiv.9.1.1978

Abstract

Visual understanding is one of the core elements of computer vision consisting of image classification, object detection, and segmentation. The system applies a multilayer process to obtain complex image and video understanding using deep learning methods to convert the images to text. Therefore, this study aimed to extract video in the form of frames followed by the application of Transformer and Inception V3 architectures to the image captioning process. The synchronization was based on Multi-task Deep Learning method developed by combining Convolutional Neural Network (CNN) system in the image area, Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) in the sentence area, Caption Content Network (CCN), and Relational Network Context (RCN). Moreover, Transformer Encoder-Decoder architecture was used in the process of labeling and determining the relationships between objects. The results of the image-to-text conversion process were determined by comparing prospective translated text with one or more references. This was achieved using accuracy and loss validation tables to provide graphical comparisons between the number of epochs and losses. The test results showed that the validation data accuracy was 70.166% while the loss was 22,648% and this showed more epoch iterations led to greater validation accuracy.Keywords— Visual Understanding, Transformer, Encoder, Decoder
Co-Authors A. A. K. Oka Sudana Adie Wahyudi Oktavia Gama Agung Udayana Putra Ahmad Catur Widyatmoko Anak Agung Ketut Agung Cahyawan Wiranatha Anak Agung Kompiang Oka Sudana Anindya Santika Devi Ariana, Anak Agung Gede Bagus Arsa, Dewa Made Sri Arya Widyaningrat, Made Gunawan Astutik, Dian Bagus Yudistira Citra Arum Sari Desak Ayu Savita Desak Ayu Sista Dewi Desy Purnami Singgih Putri Dewa Agung Krishna Arimbawa P Dewa Ayu Nadia Taradhita Dewa Made Sri Asra Dwi Putra Githa Dwi Rusjayanthi, Dwi Erdiawan Erdiawan Erdiawan Erdiawan Erdiawan Erdiawan, Erdiawan G M Arya Sasmita Gede Eridya Bayu Gede Ngurah Pasek Pusia Putra Gede Riska Wiradarma I Dewa Gede Wahya Dhiyatmika I Gede Aditya Nugraha I Gede Galang Surya Prabawa I Gede Hendra Parwata I Gede Suarjana I Gede Sujana Eka Putra I Gede Sujana Eka Putra, I Gede Sujana Eka I Gusti Ayu Agung Diatri Indradewi I Gusti Ayu Triwayuni I Gusti Made Ngurah Ardi Yasa I Gusti Ngurah Dwiva Hardijaya I Kadek Erik Priyanto I Kadek Surya Widiakumara I Ketut Adi Purnawan I Made Agus Dwi Suarjaya I Made Aris Satia Widiatmika I Made Budi Adnyana I Made Budi Sentana I MADE SUDARMA I Made Sukafona I Made Sukarsa I Made Sukarsa I Made Sunia Raharja I Made Suwija Putra I Made Suwija Putra, I Made I Made Yudha Arya Dala I N Satya Kumara I Nyoman Gede Arya Astawa I Nyoman Gunantara I Nyoman Piarsa I Nyoman Putu Suwindra I Nyoman Satria Paliwahet I Putu Adi Purnawan I Putu Agung Bayupati I Putu Agus Eka Darma Udayana I Putu Agus Eka Darma Udayana, I Putu Agus Eka I Putu Agus Eka Pratama I Putu Arya Dharmaadi I Putu Bayu Krisnawan I Putu Indra Permana I Putu Jordi Astika I Putu Satwika Putra I Putu Yoga Pertama Yasa I Wayan Agus Surya Darma I Wayan Budi Sentana I Wayan Gunaya I Wayan Muka I Wayan Ryon Waryanta I Wayan Wahyu Gautama Ida Ayu Dwi Giriantari Ida Ayu Putu Febri Imawati Ida Bagus Nyoman Yoga Ligia Prapta Kadek Adi Praptha Kadek Suar Wibawa Komang Ayu Triana Indah Komang Budiarta Lie Jasa Linawati Linawati Luki Ardiantoro M Sudarma Made Adi Widyatmika Made Sudarma Made Sudarma Made Sudarma Mimin F Rohmah Minho Jo Minho Jo Minho Jo Naser Jawas Ni Kadek Ariasih, Ni Kadek Ni Kadek Dwi Rusjayanthi, Ni Kadek Ni Kadek Riska Sadini Ni Komang Surya Cahyani Putri Ni Komang Sutiari Ni Komang Widyasanti Ni Luh Gede Pivin Suwirmayanti, S.Kom, MT, Ni Luh Gede Pivin Ni Made Ary Esta Dewi W Ni Made Ary Esta Dewi Wirastuti Ni Made Ika Marini Mandenni Ni Putu Ayu Oka Wiastini Ni Putu Chendy Widya Santi Ni Putu Intan Waindika Dharma Ni Putu Ratindia Apriyanti Ni Putu Sutramiani Nyoman Purnama, Nyoman Nyoman Putra Sastra Nyoman S Kumara Nyoman Sumerta Yasa Perdana, I Putu Iduar Pirade, Evangelika Purnamaswari, Anak Agung Arimas Putra, I Made Suwija Putri Isma Oktawiani Putu Githa Pratiwi Putu Manik Prihatini Putu Putri Wrestra Saridewi putu roy nurbhawa Putu Wira Buana Ricky Aurelius Nutanto Diaz, Ricky Aurelius Riskiyanti, Zuraida Malini Cantika Risky Aswi R, Risky Rosalia Hadi Rukmi Sari Hartati Rukmi Sari Hartati Siti Helmyati Sulya Arya Wasika Tri Ginarsa, I Nyoman Adi Wayan Oger Vihikan Wijayakusuma, I Gusti Ngurah Lanang Wira Bhuana Wira Bhuana, Wira Wiranatha, AA.Kt. Agung Cahyawan Yandi Perdana Yogiswara Dharma Putra Yudiadewi, Made Aprisintia Yusliza Binti Mohd Yasin