Claim Missing Document
Check
Articles

Loan Approval Classification Using Ensemble Learning on Imbalanced Data Anadra, Rahmi; Sadik, Kusman; Soleh, Agus M; Astari, Reka Agustia
Enthusiastic : International Journal of Applied Statistics and Data Science Volume 4 Issue 2, October 2024
Publisher : Universitas Islam Indonesia

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.20885/enthusiastic.vol4.iss2.art1

Abstract

Loan processing is an important aspect of the financial industry, where the right decisions must be made to determine loan approval or rejection. However, the issue of default by loan applicants has become a significant concern for financial institutions. Hence, ensemble learning needs to be used with random forest and Extreme Gradient Boosting (XGBoost) algorithms. Unbalanced data are handled using the Synthetic Minority Over-sampling Technique (SMOTE). This research aimed to improve accuracy and precision in credit risk assessment to reduce human workload. Both algorithms used a dataset of 4,296 with 13 variables relevant to making loan approval decisions. The research process involved data exploration, data preprocessing, data sharing, model training, model evaluation with accuracy, sensitivity, specificity, and F1-score, model selection with 10-fold cross-validation, and important variables. The results showed that XGBoost with imbalanced data handling had the highest accuracy rate of 98.52% and a good balance between sensitivity of 98.83%, specificity of 98.01, and F1-score of 98.81%. The most important variables in determining loan approval are credit score, loan term, loan amount, and annual income.
Multilevel Semiparametric Modeling with Overdispersion and Excess Zeros on School Dropout Rates in Indonesia Tarida, Arna Ristiyanti; Djuraidah, Anik; Soleh, Agus Mohamad
JTAM (Jurnal Teori dan Aplikasi Matematika) Vol 9, No 3 (2025): July
Publisher : Universitas Muhammadiyah Mataram

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.31764/jtam.v9i3.30102

Abstract

This study aims to identify key factors influencing high school dropout rates in Indonesia by applying advanced statistical modeling that accounts for complex data characteristics. Dropout data often display overdispersion (variability greater than expected) and excess zeros (many students not dropping out), which, if ignored, can bias conclusions.  To address this, we compare parametric models, Zero-Inflated Poisson Mixed Model (ZIPMM), Zero-Inflated Generalized Poisson Mixed Model (ZIGPMM), and Zero-Inflated Negative Binomial Mixed Model (ZINBMM), with their semiparametric counterparts (SZIPMM, SZIGPMM, SZINBMM). The semiparametric models use B-spline functions to capture nonlinear relationships between predictors and dropout rates, with flexibility. Model performance was evaluated using Akaike Information Criterion (AIC) and Root Mean Square Error (RMSE) across 100 simulation repetitions to ensure robustness. Results show that the semiparametric ZIGPMM (SZIGPMM) outperformed other models, achieving the lowest average AIC (18969.62), suggesting the best trade-off between model fit and complexity. The optimal spline configuration used knot point 2 and order 3, with a Generalized Cross-Validation (GCV) score of 9.4107. Key predictors of dropout include school status (public or private), student-teacher ratio, distance from home to school, parental education level, parental employment status, and number of siblings. These findings provide actionable insights for education policymakers, emphasizing the need to address structural and socioeconomic barriers to reduce dropout rates effectively.
Rotation Double Random Forest Algorithm to Predict The Food Insecurity Status of Households Rais; Agus Mohamad Soleh; Budi Susetyo
Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) Vol 8 No 1 (2024): February 2024
Publisher : Ikatan Ahli Informatika Indonesia (IAII)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29207/resti.v8i1.5540

Abstract

The ensemble tree method has been proven to handle classification problems well. The strength of the ensemble tree technique lies in the diversity and independence between each tree. Increasing the diversity of mutually independent decision trees improves the performance of the model. Various studies propose the development of ensemble tree-based models by forming algorithms that create decision trees that are formed independently of each other and have various inputs. These include random forest (RF), rotation forest (RoF), double random forest (DRF), and the latest is rotation double random forest (RoDRF). RoDRF rotates or transforms data with the intent of producing better diversity among the learning base. RoDRF applies the concept of variable rotation to trees based on the DRF algorithm. Random rotations or transformations on different feature subspaces produce different projections, leading to better generalization or prediction performance. This research aims to compare the performance of RoDRF with the RF, RoF, and DRF models on unbalanced data in cases of food insecurity. Class imbalance will be handled with two methods, namely EasyEnsemble and SMOTE-NC. The research results show that the DRF's model with EasyEnsemble techniques produces a model with the best performance among several algorithms tested. Although the resulting precision is 0.62274 and the AUC value is 0.68501, the model can predict each class equally. All algorithms with EasyEnsemble treatment have average AUC values significantly different from each other based on statistical test results. This research also used SHAP to explain variables that significantly contribute to the household's food insecurity status model.
LR-GLASSO Method for Solving Multiple Explanatory Variables of the Village Development Index Yunus, M.; Soleh, Agus M; Saefuddin, Asep; Erfiani
Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) Vol 8 No 2 (2024): April 2024
Publisher : Ikatan Ahli Informatika Indonesia (IAII)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29207/resti.v8i2.5656

Abstract

Sustainable Development Goals (SDGs) are developments that maintain sustainable improvement in society’s economic, social, and environmental welfare. Kemendes PDTT RI has issued the Village Development Index (VDI) to provide information and the status of village progress to support village development to improve the National SDGS. Modeling with multiple explanatory variables causes a high correlation between explanatory variables, multicollinearity, and coefficient estimation results, which have a large variance and overfitting in the prediction results. The modeling solution uses LASSO and GLASSO. The binary categorical response data use binary logistic regression (LR), so LR-LASSO and LR-GLASSO are used. North Maluku Province has a VDI ranking that tends to fall in 2018-2022. On the basis of the mean and variance of the coefficient estimation results and misclassification errors, LR-GLASSO is better than LR-LASSO and LR. LR-GLASSO is recommended for analyzing VDI data because it has many explanatory variables and the correlation between them is relatively high. The Indonesian government recommendation, if it is to increase the status of VDI in Indonesia, especially in the north Maluku province, is to increase the number of electricity users, food and beverage stores, and other cooperatives. The Indonesian government also needs to pay attention to villages relatively far from the regent's office, between food and beverage stalls, and supporting health centers, because they still need to be developed compared to other villages, and more than 50% of the villages are underdeveloped. If the Village SDGs are formulated by increasing the VDI status, it will support the achievement of the SDGs goals nationally.
Comparison of clustering analysis of K-means, K-medoids, and fuzzy C-means methods: case study of school accreditation in west java Hasnataeni, Yunia; Nurhambali, M Rizky; Ardhani, Rizky; Hafsah, Siti; Soleh, Agus M
Journal of Soft Computing Exploration Vol. 6 No. 2 (2025): June 2025
Publisher : SHM Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.52465/joscex.v6i2.575

Abstract

This research aims to analyze school accreditation data in West Java using clustering methods: K-Means, K-Medoids, and Fuzzy C-Means, to identify patterns and groups of schools based on similar characteristics. K-Means, known for its simplicity, suggests an optimal two-cluster solution based on silhouette values but employs three clusters for detailed analysis. K-Medoids, noted for its robustness against outliers, achieves the best clustering with a lowest Davies-Bouldin Index (DBI) of 0.8 and the highest Silhouette Information (SI) value of 0.46. Fuzzy C-Means, which assigns membership degrees to each data point across clusters, performs reasonably well with a DBI of 0.87 and an SI value of 0.40, while K-Means shows the highest DBI of 0.9 and the lowest SI value of 0.39. The findings highlight K-Medoids as the superior method for clustering. Regions with lower educational quality, such as Bekasi and Cianjur regions, require priority interventions, whereas areas with better quality, like Bandung and Bekasi regions, can serve as models. Data-driven approaches, inter-regional collaboration, and continuous monitoring and evaluation are recommended to optimize educational policies and enhance overall educational quality in West Java.
FUNCTION GROUP SELECTION OF SEMBUNG LEAVES (BLUMEA BALSAMIFERA) SIGNIFICANT TO ANTIOXIDANTS USING OVERLAPPING GROUP LASSO kusnaeni, kusnaeni; Soleh, Agus M; Afendi, Farit M; Sartono, Bagus
BAREKENG: Jurnal Ilmu Matematika dan Terapan Vol 16 No 2 (2022): BAREKENG: Jurnal Ilmu Matematika dan Terapan
Publisher : PATTIMURA UNIVERSITY

Show Abstract | Download Original | Original Source | Check in Google Scholar | Full PDF (476.663 KB) | DOI: 10.30598/barekengvol16iss2pp721-728

Abstract

Functional groups of sembung leaf metabolites can be detected using FTIR spectrometry by looking at the spectrum's shape from specific peaks that indicate the type of functional group of a compound. There were 35 observations and 1866 explanatory variables (wavelength) in this study. The number of explanatory variables more than the number of observations is high-dimensional data. One method that can be used to analyze high-dimensional data is penalized regression. The overlapping group lasso method is a development of the group-based penalized regression method that can solve the problem of selecting variable groups and members of overlapping groups of variables. The results of selecting the variable groups using the overlapping group lasso method found that the functional groups that were significant for the antioxidants of sembung leaves were C=C Unstructured, CN amide, Polyphenol, Sio2.
TEXT CLUSTERING ONLINE LEARNING OPINION DURING COVID-19 PANDEMIC IN INDONESIA USING TWEETS Tyas, Maulida Fajrining; Kurnia, Anang; Soleh, Agus Mohamad
BAREKENG: Jurnal Ilmu Matematika dan Terapan Vol 16 No 3 (2022): BAREKENG: Journal of Mathematics and Its Applications
Publisher : PATTIMURA UNIVERSITY

Show Abstract | Download Original | Original Source | Check in Google Scholar | Full PDF (1033.901 KB) | DOI: 10.30598/barekengvol16iss3pp939-948

Abstract

To prevent the spread of corona virus, restriction of social activities are implemented including school activities which reaps the pros and cons in community. Opinions about online learning are widely conveyed mainly on Twitter. Tweets obtained can be used to extract information using text clustering to group topics about online learning during pandemic in Indonesia. K-Means is often used and has good performance in text clustering area. However, the problem of high dimensionality in textual data can result in difficult computations so that a sampling method is proposed. This paper aims to examine whether a sampling method to cluster tweets can result to an efficient clustering than using the whole dataset. After pre-processing, five sample sizes are selected from 28300 tweets which are 250, 500, 2500, 10000 and 20000 to conduct K-Means clustering. Results showed that from 10 iterations, three main cluster topics appeared 90%-100% in sample size of 2500, 10000 and 20000. Meanwhile sample size of 250 and 500 tend to produced 20%-60% appearance of the three main cluster topics. This means that around 8% to 35% of tweets used can yield representative clusters and efficient computation which is four times faster than using entire dataset.
Robust Continuum Regression Study of LASSO Selection and WLAD LASSO on High-Dimensional Data Containing Outliers Daulay, Nurmai Syaroh; Erfiani, Erfiani; Soleh, Agus M
JTAM (Jurnal Teori dan Aplikasi Matematika) Vol 8, No 3 (2024): July
Publisher : Universitas Muhammadiyah Mataram

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.31764/jtam.v8i3.23123

Abstract

In research, we often encounter problems of multicollinearity and outliers, which can cause coefficients to become unstable and reduce model performance. Robust Continuum Regression (RCR) overcomes the problem of multicollinearity by reducing the number of independent variables, namely compressing the data into new variables (latent variables) that are independent of each other and whose dimensions are much smaller and applying robust regression techniques so that the complexity of the regression model can be reduced without losing essential information from data and provide more stable parameter estimates. However, it is hampered in the computational aspect if the data has very high dimensions (p>>n). In the initial stage, it is necessary to reduce dimensions by selecting variables. The Least Absolute Shrinkage and Selection Operator (LASSO) can overcome this but is sensitive to the presence of outliers, which can result in errors in selecting significant variables. Therefore, we need a method that is robust to outliers in selecting explanatory variables such as Weighted Least Absolute Deviations with LASSO penalty (WLAD LASSO) in selecting variables by considering the absolute deviation of the residuals. This method aims to overcome the problem of multicollinearity and model instability in high-dimensional data by paying attention to resistance to outliers. Leverages the outlier resistant RCR and variable selection capabilities of LASSO and WLAD LASSO to provide a more reliable and efficient solution for complex data analysis. Measure the performance of RKR-LASSO and RKR-WLAD LASSO; simulations were carried out using low-dimensional data and high-dimensional data with two scenarios, namely without outliers (δ= 0%) and with outliers (δ= 10%, 20%, 30%) with a level of correlation (ρ = 0.1,0.5,0.9). The analysis stage uses RStudio version 4.1.3 software using the "MASS" package to generate data that has a multivariate normal distribution, the "glmnet" package for LASSO variable selection, the "MTE" package for WLAD LASSO variable selection. The simulation results show the performance of RKR-LASSO tends to be superior in terms of model goodness of fit compared to RKR-WLAD LASSO. However, the performance of RKR-LASSO tends to decrease as outliers and correlations increase. RKR-LASSO tends to be looser in selecting relevant variables, resulting in a simpler model, but the variables chosen by LASSO are only marginally significant. RKR-WLAD LASSO is stricter in variable selection and only selects significant variables but ignores several variables that have a small but significant impact on the model.
Comparison of LASSO, Ridge, and Elastic Net Regularization with Balanced Bagging Classifier Nisrina Az-Zahra, Putri; Sadik, Kusman; Suhaeni, Cici; Mohamad Soleh, Agus
Parameter: Jurnal Matematika, Statistika dan Terapannya Vol 4 No 2 (2025): Parameter: Jurnal Matematika, Statistika dan Terapannya
Publisher : Jurusan Matematika FMIPA Universitas Pattimura

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.30598/parameterv4i2pp287-296

Abstract

Predicting Drug-Induced Autoimmunity (DIA) is crucial in pharmaceutical safety assessment, as early identification of compounds with autoimmune risk can prevent adverse drug reactions and improve patient outcomes. Classification analysis often faces challenges when the number of predictor variables exceeds the number of observations or when high correlations among predictors lead to multicollinearity and overfitting. Regularization methods, such as Ridge Regression, Least Absolute Shrinkage and Selection Operator (LASSO), and Elastic-Net, help stabilize parameter estimation and improve model interpretability. This study focuses on building a binary classification model to predict the risk of DIA using 196 molecular descriptors derived from chemical compound structures. To address class imbalance in the response variable, the Balanced Bagging Classifier (BBC) is combined with regularized logistic regression models. Elastic Net + BBC outperforms other models with the highest accuracy (0.825), followed closely by LASSO + BBC and Ridge + BBC (both 0.816). This integration not only improves classification accuracy but also enhances generalization and the reliable detection of minority class instances, supporting the early identification of autoimmune risks in drug discovery.
EVALUATING RANDOM FOREST AND XGBOOST FOR BANK CUSTOMER CHURN PREDICTION ON IMBALANCED DATA USING SMOTE AND SMOTE-ENN Andespa, Reyuli; Sadik, Kusman; Suhaeni, Cici; Soleh, Agus M
MEDIA STATISTIKA Vol 18, No 1 (2025): Media Statistika
Publisher : Department of Statistics, Faculty of Science and Mathematics, Universitas Diponegoro

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.14710/medstat.18.1.25-36

Abstract

The banking industry faces significant challenges in retaining customers, as churn can critically affect both revenue and reputation. This study introduces a robust churn prediction framework by comparing the performance of XGBoost and Random Forest algorithms under imbalanced data conditions. The novelty of this research lies in integrating the SMOTE and SMOTE-ENN techniques with machine learning algorithms to enhance model performance and reliability on highly imbalanced datasets. Unlike conventional approaches that rely solely on oversampling or undersampling, this study demonstrates that the hybrid combination of XGBoost and SMOTE provides superior predictive accuracy, stability, and efficiency. Hyperparameter optimization using GridSearchCV was conducted to identify the most effective parameter configurations for both algorithms. Model performance was evaluated using the F1-Score and Area Under the Curve (AUC). The results indicate that XGBoost with SMOTE achieved the best performance, with an F1-Score of 0.8730 and an AUC of 0.9828, showing an optimal balance between precision and recall. Feature importance analysis identified Months_Inactive_12_mon, Total_Trans_Amt, and Total_Relationship_Count as the most influential predictors. Overall, this approach outperforms traditional resampling and modeling techniques, providing practical insights for data-driven customer retention strategies in the banking industry.
Co-Authors Abd Rahman Abialam Koesnandy H Ade Ariyo Yudanto Afendi, Farit M Aji Hamim Wigena Akbar Rizki Alfa Nugraha Pradana Amir Abduljabbar Dalimunthe Anadra, Rahmi Anang Kurnia Andespa, Reyuli Andriansyah, . Anik Djuraidah Annisarahmi Nur Aini Aldania Ardhani, Rizky Arif Handoyo Marsuhandi Aris Yaman ASEP SAEFUDDIN Astari, Reka Agustia Baehera, Seta Bagus Sartono Belinda, Nadira Sri Budi Susetyo Cici Suhaeni Cici Suhaeni Cici Suhaeni Daulay, Nurmai Syaroh Dede Dirgahayu Domiri Dede Dirgahayu Domiri Dede Dirgahayu Domiri, Dede Dirgahayu Deri Siswara Devi Andrian Erfiani Erfiani Erfiani Erfiani Erfiani Erfiani Etis Sunandi Farit Mochamad Afendi Fitri Mudia Sari Fitrianto, Anwar Fulazzaky, Tahira Gede Awantara Hamim Wigena, Aji Hari Wijayanto Hasnataeni, Yunia Hengki Muradi Herlin Fransiska I Gusti Ngurah Sentana Putra I Gusti Ngurah Sentana Putra I Made Sumertajaya Indahwati Indra Rivaldi Siregar Inria Purwaningsih Jumansyah, L. M. Risman Dwi Jumansyah, L.M. Risman Dwi Karel Fauzan Hakim Khairil Anwar Notodiputro Khairil Anwar Notodiputro Kusman Sadik Kusnaeni Kusnaeni, Kusnaeni Latifah K. Darusman Leni Anggraini Susanti Lisa Amelia M. Yunus Meavi Cintani Mohamad Rafi Mubarak, Fadhlul Muhammad Nur Aidi Muhammad Nur Aidi Muhammad Nuruddin Prathama Muradi, Hengki Naila Nabiha Qonita Naufalia Alfiryal Nisrina Az-Zahra, Putri Nur Aulia Maknunah Nurhambali, M Rizky Nurizki, Anisa Pika Silvianti Rahardiantoro, Septian Rais Ramadhani, Dini Rizki Manaf, Silmi Anisa Rochman, Nur Sabrina Adnin Kamila Sabrina Adnin Kamila Sachnaz Desta Oktarina Sentana Putra I Gusti Ngurah Seran, Karlina Setyono Siti Arni Wulandya, Siti Arni Siti Hafsah Suhaeni, Cici Tarida, Arna Ristiyanti Tyas, Maulida Fajrining Uswatun Hasanah Uswatun Hasanah Utami Dyah Syafitri Yudistira Yudistira Yumna Karimah Zamrah Mutmainah _ Aunuddin