JOURNAL OF INFORMATION SYSTEM RESEARCH (JOSH)
Vol 6 No 1 (2024): Oktober 2024

Evaluating the Effectiveness of Machine Learning Models for Cyberattack Detection: A Study on Model Generalization and Dataset Imbalance

Airlangga, Gregorius (Unknown)



Article Info

Publish Date
31 Oct 2024

Abstract

In today's rapidly evolving digital landscape, detecting and preventing cyberattacks has become crucial for securing networks and data. This study evaluates the performance of several machine learning models, including RandomForest, GradientBoosting, XGBoost, LightGBM, CatBoost, Support Vector Classifier (SVC), Logistic Regression, and an ensemble Voting Classifier, in detecting and classifying cyberattacks. The models were tested on a real-world cybersecurity dataset with significant class imbalance, where benign traffic vastly outnumbers malicious attacks. Results showed that while some models, such as RandomForest and the Voting Classifier, achieved high training accuracy, they suffered from overfitting, with test accuracies not exceeding 34%. Boosting models like XGBoost and LightGBM exhibited better generalization than RandomForest but still struggled to handle the dataset complexity. The primary limitations of this study include the dataset's imbalance, the high dimensionality of the features, and the models’ tendency to overfit. These challenges highlight the need for more robust data preprocessing techniques, hyperparameter tuning, and exploration of advanced models, such as deep learning architectures, for future work. The findings provide insights into the challenges of using machine learning for cybersecurity attack detection and point toward future directions for improving model performance in real-world settings.

Copyrights © 2024






Journal Info

Abbrev

josh

Publisher

Subject

Computer Science & IT Decision Sciences, Operations Research & Management

Description

Artikel yang dimuat melalui proses Blind Review oleh Jurnal JOSH, dengan mempertimbangkan antara lain: terpenuhinya persyaratan baku publikasi jurnal, metodologi riset yang digunakan, dan signifikansi kontribusi hasil riset terhadap pengembangan keilmuan bidang teknologi dan informasi. Fokus Journal ...