<![CDATA[Machine learning models deployed in operational settings rarely encounter data identically distributed to their training set. Shifts in population composition, measurement processes, and sampling frames routinely cause performance degradation, undermining both accuracy and trust. This study empirically examines model stability and generalization under controlled distribution shift using the UCI Adult/Census Income dataset (48,842 records, 14 features). Four representative classifiers-Logistic Regression, XGBoost, LightGBM, and CatBoost-were trained and evaluated across three scenarios: an in-distribution stratified random split, a demographic shift in which the model is trained on individuals under 40 years old and tested on those aged 40 and above, and a structural subpopulation shift in which the model is trained on non-degree holders and tested on degree holders. Contrary to the conventional expectation that distribution shift monotonically degrades performance, the empirical F1-score results reveal a more nuanced picture: all four classifiers actually achieved higher F1-scores on the education-shifted test set than on the in-distribution baseline, with Logistic Regression gaining +0.145 F1 points. This counter-intuitive outcome is driven by the increased positive-class prior in the shifted target distributions. When stability is operationalized as the signed average F1 change (with rank 1 assigned to the smallest, i.e. most negative, value), Logistic Regression ranked first (average change −0.076), followed by CatBoost (−0.016), LightGBM (−0.013), and XGBoost (−0.013); we show, however, that under the operationally meaningful absolute-change criterion this ordering reverses and the gradient boosting models are the most stable. However, accuracy tells a contrasting story: Logistic Regression's accuracy fell by 16.4 percentage points under the age shift, whereas the gradient boosting models retained accuracy above 0.81. These findings demonstrate that single-metric stability evaluation is misleading and that shift robustness must be characterized through multiple complementary metrics.]]>
Copyrights © 2026
