Facial Expression Recognition inCovid-19 Pandemic EraUsing 2 Stage Convolutional Neural Network

Kevin Kurniawan (Bina Nusantara University, Jakarta)
Lili Ayu Wulandhari (Bina Nusantara University, Jakarta)
Antonius Rildo Pramudya Gondosiswojo (Bina Nusantara University, Jakarta)

Jurnal Sistem Komputer dan Informatika (JSON)
Vol 5, No 1 (2023): September 2023

In the Covid-19 pandemic era, the use of face mask has become mandatory for all citizens to prevent the spread of the virus. This regulation has becomes a big problem for the Facial Expression Recognitions (FER) applications because face mask cover more than halfof human faces. Starting from this problem, this experiment attemps to produce a robust network which can perform well in both conditions: recognizing expressions with and without a face mask. Dataset used in this experiment is FER2013, with a preprocessing step to produce a FER2013 masked. This research uses 2-stage network, where the first network is used to recognize whether the subject is wearing a mask or not, and then the second network is used to recognize the expression based on the result from the first stage. The network in this experiment is based on Convolutional Neural Network (CNN), with Imagenet as our pretrainedmodel and EfficientNet as our architecture model.Ourproposed model has shown quite good performance in recognizing expressions, even when the data consists of subjects who use and do not use masks with an accuracy of 57.55%.