The Indonesian Journal of Computer Science
Vol. 14 No. 3 (2025): The Indonesian Journal of Computer Science

AI-Enhanced Multi-Modal Emotion and Personalized Responding System for Undergraduates

Karunathilaka, Chamoda (Unknown)
De Vass Gunawardane, Dilun (Unknown)
Athuluwage, Tharaka (Unknown)
Marasinghe, Chamalka (Unknown)
Vidanaralage, Anjana Junius (Unknown)
Fernando, Harinda (Unknown)



Article Info

Publish Date
09 Jun 2025

Abstract

Undergraduate students face increasing academic and personal pressures, often leading to stress and emotional distress. Traditional single-modal emotion recognition systems, relying solely on facial or vocal analysis, struggle with accuracy due to environmental variations and limited contextual awareness. This research proposes a multi-modal AI-driven emotion recognition system that integrates facial and vocal data for enhanced real-time emotional detection and response. The system leverages Vision Transformers (ViTs) for facial feature extraction and Mel-Frequency Cepstral Coefficients (MFCC) for speech-based emotion analysis, ensuring improved classification through confidence-weighted temporal fusion. Additionally, an adaptive response generation module utilizes natural language processing (NLP) and text-to-speech (TTS) synthesis for human-like interactions. To enable scalable mobile deployment, the model is optimized with quantized lightweight transformers, achieving sub 300ms inference latency. Bias mitigation techniques ensure fairness across demographic groups. This research contributes to affective computing, human-computer interaction, and AI-driven emotional intelligence, offering a scalable and ethically responsible solution for virtual counseling, AI-assisted tutoring, and mental health support.

Copyrights © 2025






Journal Info

Abbrev

ijcs

Publisher

Subject

Computer Science & IT Electrical & Electronics Engineering Engineering

Description

The Indonesian Journal of Computer Science (IJCS) is a bimonthly peer-reviewed journal published by AI Society and STMIK Indonesia. IJCS editions will be published at the end of February, April, June, August, October and December. The scope of IJCS includes general computer science, information ...