Journal of Applied Data Sciences
Vol 7, No 1: January 2026

Multimodal AI Framework for Sign Language Recognition and Medical Informatics in Hearing-Impaired Patients

Nuankaew, Pratya (Unknown)
Khamthep, Parin (Unknown)
Jaitem, Patdanai (Unknown)
Nuankaew, Kuljira S. (Unknown)
Nuankaew, Kaewpanya S. (Unknown)
Nuankaew, Wongpanya S. (Unknown)



Article Info

Publish Date
14 Jan 2026

Abstract

This study assesses the feasibility of YOLO-based detectors for the recognition of Thai Sign Language (TSL) within clinical intake workflows. We benchmark YOLOv5 through YOLOv10 over 100 to 150 training epochs and evaluate metrics including Precision, Recall, mAP@50, mAP@50:95, alongside training and validation losses to gauge stability. The losses decrease steadily as detection metrics improve; YOLOv10 offers the optimal balance, with Precision at 0.953, Recall at 0.939, mAP@50 at 0.933, and mAP@50:95 at 0.492. The improvements observed at stricter IoU thresholds are modest, underscoring ongoing challenges in achieving accurate localization and generalization across varying lighting conditions, viewpoints, occlusions, and motion. YOLOv11 has been excluded from the primary results due to abnormal loss behavior. These findings endorse a multimodal pipeline that employs an image-based detector as the central perception component, supplemented with pose and key point cues, as well as OCR and NLP layers, to transform recognized signs into structured medical intents for triage and telemedicine applications. Future research will focus on expanding sequence-level evaluation, incorporating dialects and co-articulation in TSL, and developing compressed or distilled models to facilitate reliable on-device inference in resource-constrained environments. 

Copyrights © 2026






Journal Info

Abbrev

JADS

Publisher

Subject

Computer Science & IT Control & Systems Engineering Decision Sciences, Operations Research & Management

Description

One of the current hot topics in science is data: how can datasets be used in scientific and scholarly research in a more reliable, citable and accountable way? Data is of paramount importance to scientific progress, yet most research data remains private. Enhancing the transparency of the processes ...