<![CDATA[The high rate of order cancellations in marketplaces is a challenge that can impact operational efficiency, inventory management, and customer service quality. The ability to identify orders potentially canceled early is crucial so businesses can take more appropriate mitigation measures. The purpose of this study is to compare the performance of the Random Forest, and XGBoost algorithms in the marketplace order cancellations prediction process and to interpret the factors that influence predictions using SHAP (SHapley Additive exPlanations)-based explainable machine learning. The study used a marketplace transaction dataset that had undergone data cleaning, feature transformations, and class through the application of the Synthetic Minority Oversampling Technique (SMOTE) method. Next, the model performance is evaluated using the Accuracy, Precision, Recall, and F1-Score metrics. Four model scenarios were tested: Random Forest + SMOTE, Random Forest Tuned + SMOTE, XGBoost + SMOTE, and XGBoost Tuned + SMOTE. Based on the research results, it is indicated that the XGBoost Tuned + SMOTE model produces the best performance with accuracy values reaching 86.62%, precision reaching 51.43%, recall reaching 34.95%, and F1-Score reaching 41.62%. Meanwhile, the XGBoost + SMOTE model produced the highest recall of 37.09% with an F1-Score of 41.48%. Considering that the research objective is to detect potentially canceled orders in imbalance data, the XGBoost + SMOTE model was chosen as the best model because it produced the highest recall of 37,09% with a competitive F1-Score of 41,48%, making it more capable of detecting minority classes compared to other models. SHAP analysis showed that several transaction features contribute more dominantly to the prediction results produced by the model. These findings can be used as a basis for decision-making to reduce the risk of order cancellations and improve the effectiveness of marketplace operations.]]>
