Design of a Hybrid SVM Ensemble and Large Language Model Chatbot for Multi-Class Intent Classification in Clinic Information Services - Vina Rahmadiany; Lindawati Lindawati; Aryanti Aryanti (Bulletin of Network Engineer and Informatics, 2026)
Clinical information service chatbots require an accurate multiclass intent classification mechanism to handle informal language variations, medical abbreviations, service-related inquiries, and health complaints in Indonesian. This study aims to develop a hybrid chatbot architecture that integrates a Support Vector Machine ensemble for intent classification with Meta-Llama-3.1-8B-Instruct to generate relevant, natural, and context-aware responses. The dataset consisted of 3,293 utterance patterns across 66 intent classes. The proposed approach employed an 80:20 stratified split; semantic augmentation on the training data; preprocessing via abbreviation normalization, stopword removal, and Sastrawi stemming; TF-IDF feature extraction using unigram and bigram word n-grams and character n-grams; chi-squared feature selection; and hyperparameter optimization via grid search. The classification model was constructed by combining three calibrated LinearSVC classifiers via probability-based soft voting. Experimental results achieved an accuracy of 89.83%, a weighted F1-score of 89.90%, a kappa of 0.8966, a macro AUC of 0.9942, and an average response time of 61.87 ms. McNemar's test indicated statistically significant improvements over Complement Naïve Bayes, Logistic Regression, and Decision Tree, while no significant difference was observed compared with Single SVM. Therefore, the proposed architecture is effective at supporting multiclass intent classification and delivering fast, relevant chatbot responses for clinical information services.