Improving Early Detection of Cervical Cancer Through Deep Learning-Based Pap Smear Image Classification

Merlina, Nita
Prasetio, Arfhan
Zuniarti, Ida
Mayangky, Nissa Almira
Sulistyowati, Daning Nur
Aziz, Faruq

Journal of Applied Data Sciences
Vol 6, No 2: MAY 2025

Cervical cancer is one of the leading causes of death in women worldwide, making early detection of the disease crucial. This study proposes a deep learning-based approach that has the advantage of leveraging pre-trained models to save data, time, and computation to classify Pap smear images without relying on segmentation, which is traditionally required to isolate key morphological features. Instead, this method leverages deep learning to identify patterns directly from raw images, reducing preprocessing complexity while maintaining high accuracy. The dataset used in this study is a public data repository from Nusa Mandiri University (RepomedUNM), which has a wider variety of data. This dataset is used to classify images into four categories: Normal, LSIL, HSIL, and Koilocytes. The dataset consists of 400 images evenly distributed, ensuring class balance during training. Transfer learning is applied using five Convolutional Neural Network (CNN) architectures: ResNet152V2, InceptionV3, ResNet50V2, DenseNet201, and ConvNeXtBase. To prevent overfitting, techniques such as data augmentation, dropout regularization, and class weight adjustment are applied. The evaluation results in this study showed the highest accuracy with a value of ResNet152V2 = 0.9025, InceptionV3 = 0.8953 and DenseNet201 = 0.8845. ResNet152V2 excelled in extracting complex features, while InceptionV3 showed better computational efficiency. The study also highlighted the clinical impact of misclassification between Koilocytes and LSIL, which may affect diagnostic outcomes. Data augmentation techniques, including horizontal and vertical flipping and normalization, improved the model's generalization to a wide variety of images. Specificity was emphasized as a key evaluation metric to minimize false positives, which is important in medical diagnostics. The findings confirmed that transfer learning effectively overcomes the limitations of small datasets and improves the classification accuracy of pap smear images. This approach shows potential for integration into clinical workflows to enable automated and efficient cervical cancer detection.