JOURNAL OF APPLIED INFORMATICS AND COMPUTING
Vol. 10 No. 5 (2026): October 2026

Toward Objective Mental Health Screening via Facial Expression Analysis: A Hybrid Deep Learning Proof-of-Concept

Setyowati, Endah (Unknown)
Fauzi, Ahmad (Unknown)
Muhammad Suranegara, Galura (Unknown)
Sulaiman, Mohd Herwan (Unknown)
Fallah Ramadhani, Fauzzi (Unknown)
Raditiya Fadillah, Sahra (Unknown)
Ursula Zahra, Duke (Unknown)



Article Info

Publish Date
18 Oct 2026

Abstract

Depression remains among the largest contributors to global disease burden, yet early screening in Indonesia still relies heavily on self-report instruments prone to subjective bias. This study presents "Beyond Expression," a hybrid architecture combining a Convolutional Neural Network (ResNet-50), a Swin Transformer, and a Spatiotemporal Attention Module (TSAM) to detect depression-related facial patterns from anonymized 3D facial landmark sequences, intended as a first step toward future objective screening tools rather than a clinically validated instrument. Evaluated via 5-fold stratified cross-validation on 142 participants, the model achieved an average accuracy of 66.95% ± 5.49% and macro F1-score of 0.5102 ± 0.0969. Against a logistic regression baseline (F1 = 0.5091), a paired Wilcoxon signed-rank test and paired t-test found no statistically detectable difference, though the small fold count (n = 5) limits test power. Sensitivity remained low (24.17% ± 20.48%), and a cross-frame Grad-CAM analysis across four representative participants showed Healthy-class activation staying relatively stable across frames (correlation up to 0.80), while Depressed-class activation was markedly less consistent (correlation near zero) and partially collapsed in one case, closely matching the model's low sensitivity. A same-dataset held-out check on withheld DAIC-WOZ test participants (N = 47) gave a single-run sensitivity of 57.14% and specificity of 72.73%, higher than most cross-validation folds but not yet confirmed stable across runs. The authors treat these results as genuine findings from a proof-of-concept stage, distinct from a validated clinical diagnosis, meant to guide refinements in data collection and model design for future iterations ahead of clinical validation.

Copyrights © 2026






Journal Info

Abbrev

JAIC

Publisher

Subject

Computer Science & IT

Description

Journal of Applied Informatics and Computing (JAIC) Volume 2, Nomor 1, Juli 2018. Berisi tulisan yang diangkat dari hasil penelitian di bidang Teknologi Informatika dan Komputer Terapan dengan e-ISSN: 2548-9828. Terdapat 3 artikel yang telah ditelaah secara substansial oleh tim editorial dan ...