Article Per Year (5 Year)
p-Index From 2021 - 2026
0.408
P-Index
This Author published in this journals
All Journal Dewantara Journal of Technology Journal Artificial: Informatika dan Sistem Informasi
Muhammad Abrar Rayhan
Telkom University
Agriculture, Biological Sciences & Forestry Automotive Engineering Civil Engineering, Building, Construction & Architecture Computer Science & IT Electrical & Electronics Engineering

Published : 2 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 2 Documents
Search

 1 
Analyzing Model Stability and Generalization under Distribution Shift in Real-World Machine Learning Applications Muhammad Abrar Rayhan; Citra Feby Dermawaty Manik; Angga Prasetya Putra
Dewantara Journal of Technology Vol. 6 No. 2 (2026)
Publisher : Akademi Teknologi Industri Dewantara Palopo

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.59563/djtech.v6i2.367

Abstract

Machine learning models deployed in operational settings rarely encounter data identically distributed to their training set. Shifts in population composition, measurement processes, and sampling frames routinely cause performance degradation, undermining both accuracy and trust. This study empirically examines model stability and generalization under controlled distribution shift using the UCI Adult/Census Income dataset (48,842 records, 14 features). Four representative classifiers-Logistic Regression, XGBoost, LightGBM, and CatBoost-were trained and evaluated across three scenarios: an in-distribution stratified random split,  a demographic shift in which the model is trained on individuals under 40 years old and tested on those aged 40 and above, and a structural subpopulation shift in which the model is trained on non-degree holders and tested on degree holders. Contrary to the conventional expectation that distribution shift monotonically degrades performance, the empirical F1-score results reveal a more nuanced picture: all four classifiers actually achieved higher F1-scores on the education-shifted test set than on the in-distribution baseline, with Logistic Regression gaining +0.145 F1 points. This counter-intuitive outcome is driven by the increased positive-class prior in the shifted target distributions. When stability is operationalized as the signed average F1 change (with rank 1 assigned to the smallest, i.e. most negative, value), Logistic Regression ranked first (average change −0.076), followed by CatBoost (−0.016), LightGBM (−0.013), and XGBoost (−0.013); we show, however, that under the operationally meaningful absolute-change criterion this ordering reverses and the gradient boosting models are the most stable. However, accuracy tells a contrasting story: Logistic Regression's accuracy fell by 16.4 percentage points under the age shift, whereas the gradient boosting models retained accuracy above 0.81. These findings demonstrate that single-metric stability evaluation is misleading and that shift robustness must be characterized through multiple complementary metrics.
Implementation of Naive Bayes and Support Vector Machine for SMS Spam Classification Using the SMS Spam Collection Dataset Muhammad Abrar Rayhan
Journal Artificial: Informatika dan Sistem Informasi Vol. 4 No. 2 (2026): Oktober 2026
Publisher : Pustaka Digital Indonesia

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.54065/artificial.1249

Abstract

Short Message Service (SMS) is one of the most popular communication services on mobile networks. The rapid proliferation of mobile communication has led to an increase in spam messages offering ads, false links, and misinformation, which could pose a threat to user privacy. Automated spam detection using machine learning methods has become a key approach to tackling this problem in recent years. The aim of this research is to apply and train on how the SMS Spam Collection. Dataset for SMS spam classification using 2 machine learning algorithms, Naïve Bayes and Support Vector Machine (SVM). Several steps are taken, including data preprocessing, text cleanup, feature extraction using the Term Frequency–Inverse Document Frequency (TF-IDF) method, and model training. The performance of the implemented models is assessed using accuracy, precision, recall, F1-score, a confusion matrix, and cross-validation. The results from the experiments show that both algorithms can successfully classify these SMS spam messages. However, the Support Vector Machine model outperforms the Naïve Bayes model, achieving an accuracy of nearly 98% on the classification task. These results demonstrate that machine learning techniques, including Support Vector Machine in combination with TF-IDF feature extraction, provide reliable performance for SMS spam detection, and could be helpful for an automated filter system in m-commerce services.
Co-Authors Angga Prasetya Putra Citra Feby Dermawaty Manik