IAES International Journal of Artificial Intelligence (IJ-AI)
Vol 14, No 6: December 2025

Impact of smoothing techniques for text classification: implementation in hidden Markov model

Mathivanan, Norsyela Muhammad Noor (Unknown)
Mohd Janor, Roziah (Unknown)
Abd Razak, Shukor (Unknown)
Md. Ghani, Nor Azura (Unknown)



Article Info

Publish Date
01 Dec 2025

Abstract

A hidden Markov model (HMM) is widely used for sequence modeling in various text classification tasks. This study investigates the impact of different smoothing techniques, such as Laplace, absolute discounting, and Gibbs sampling on HMM performance across three distinct domains: e-commerce products, spam filtering, and occupational data mining. Through the comparative analysis, Laplace smoothing consistently outperforms other techniques in handling zero-probability issues, demonstrating superior performance in the e-commerce and SMS spam datasets. The HMM without any smoothing technique achieved the best results for job title classification. This divergence underscores the dataset-specific nature of smoothing requirements, where the simplicity of parameter estimation proves effective in contexts characterized by a limited and repetitive vocabulary. Hence, the findings suggest that tailored smoothing strategies are crucial for optimizing HMM performance in different textual analysis applications.

Copyrights © 2025






Journal Info

Abbrev

IJAI

Publisher

Subject

Computer Science & IT Engineering

Description

IAES International Journal of Artificial Intelligence (IJ-AI) publishes articles in the field of artificial intelligence (AI). The scope covers all artificial intelligence area and its application in the following topics: neural networks; fuzzy logic; simulated biological evolution algorithms (like ...