Optimized Machine Learning Approach for Malware Detection using Bayesian Optimization - Hilman Singgih Wicaksana; Khairul Huda (ISB Atma Luhur, 2026)
Information technology has facilitated innovation in numerous sectors and enhanced operational efficiency as a result of its rapid expansion. However, these technical innovations have also resulted in an increased likelihood of cyberattacks, particularly those that are initiated by malware. Sophisticated evasion techniques, such as polymorphic and metamorphic transformations, are frequently implemented by contemporary malware, which significantly reduces the reliability of conventional detection methods. This investigation endeavors to evaluate and contrast the efficacy of a variety of machine learning algorithms that have been enhanced by Bayesian Optimization, a probabilistic method for hyperparameter tuning that effectively identifies the optimal model configurations. The study investigates five algorithms: Multilayer Perceptron, Random Forest, Support Vector Machine, Extreme Gradient Boosting, and Hist Gradient Boosting, using a supervised learning methodology with labeled data. The performance of each model was evaluated based on its accuracy, precision, recall, and F1-score, while its optimal parameters were meticulously fine-tuned. Additionally, experiments were implemented on a dataset consisting of 58,596 records that underwent rigorous cleansing and preprocessing. The Multilayer Perceptron demonstrated the highest performance, obtaining 99.97% across all evaluation measures, according to the results. These discoveries underscore the efficiency, precision, and adaptability of refined machine learning models in detecting malware in response to the evolution of cyber threats.