Evaluating Contextual Embedding Models for Multi-Label PICO Classification in Heart Disease: Addressing the Intervention - Comparison Bottleneck - Taslim, Taslim; Handayani, Susi; Walhidayat, Walhidayat; Toresa, Dafwen (Publisher: Fakultas Ilmu Komputer, Institution: Universitas Lancang Kuning, 2025)
Accurate extraction of Population, Intervention, Comparison, and Outcome (PICO) elements from clinical texts is essential for supporting evidence-based medicine, particularly in cardiology where clinical data complexity presents significant challenges. This study investigates the comparative effectiveness of three contextual embedding models—BioBERT, PubMedBERT, and SciBERT—integrated with a Bidirectional Long Short-Term Memory (BiLSTM) architecture for multi-label PICO element classification on heart disease datasets. The experimental framework involved pre-processing clinical sentences, transforming them into contextual embeddings, and classifying PICO elements using BiLSTM-based sequence modeling. Evaluation was conducted using five key metrics: accuracy, precision, recall, F1-score, and hamming loss, supplemented by confusion matrix analysis for each PICO element. Results demonstrate that the BioBERT-BiLSTM model achieved superior performance, with an accuracy of 73.89%, F1-score of 78.54%, precision of 81.60%, and recall of 76.64%. PubMedBERT-BiLSTM exhibited the highest precision (84.12%) but lower recall, while SciBERT-BiLSTM produced slightly inferior results overall. These findings confirm the importance of using domain-specific embeddings, particularly models pre-trained on biomedical corpora, to improve classification accuracy in specialized clinical text tasks. This study concludes that the BioBERT-BiLSTM combination offers a reliable approach for automated PICO element extraction in the cardiology domain, contributing to the development of more accurate and efficient clinical decision-support systems