International Journal of Advances in Artificial Intelligence and Machine Learning
Vol. 3 No. 2 (2026): International Journal of Advances in Artificial Intelligence and Machine Learni

Learning from Heterogeneous Label Quality: Strategies for Robust Model Training

Wu Shukun (Guangxi Science & Technology Normal University)
Hafiz Muhammad Kurniawan (INTI International University)
Adolf Asih Suprianto (Politeknik Enjinering Indorama)



Article Info

Publish Date
28 Sep 2026

Abstract

Background: Training data collected from crowdsourcing platforms, automated labeling systems, and multiple annotators often contain labels with unequal levels of reliability. Conventional machine-learning methods generally treat all labels as equally trustworthy, which can reduce model performance when label errors vary across samples and annotation sources.Aims: This study proposes a quality-aware training method that explicitly models the heterogeneity of label reliability. It examines whether sample-level quality weighting can improve predictive performance and robustness under synthetic and real-world label-noise conditions.Methods: The proposed method combines annotation or neighborhood agreement, loss-based confidence, and source-level reliability to estimate a continuous quality score for each training sample. The scores are incorporated into a weighted cross-entropy objective and periodically updated during training. Experiments were conducted on CIFAR-10, CIFAR-100, CIFAR-10N, and CIFAR-100N using ResNet-18. The method was compared with cross-entropy, label smoothing, generalized cross-entropy, Co-teaching, and DivideMix.Result: Under high heterogeneous noise, the proposed method achieved accuracies of 83.21% on CIFAR-10 and 54.62% on CIFAR-100, compared with 73.48% and 45.92% for conventional cross-entropy. It also achieved the highest accuracy under the evaluated real-world noisy-label conditions. Ablation and sensitivity analyses showed that the combined quality indicators were complementary and that the method remained effective under moderate quality-estimation errors.Conclusion: Explicitly modeling label reliability improves robustness when training data contain unequal annotation quality. The proposed method provides a practical and computationally efficient strategy for learning from crowdsourced, weakly supervised, and multi-source datasets.

Copyrights © 2026






Journal Info

Abbrev

ijaaiml

Publisher

Subject

Computer Science & IT

Description

The International Journal of Advances in Artificial Intelligence and Machine Learning (IJAAIML) is a prominent academic journal dedicated to publishing cutting-edge research and developments in the fields of Artificial Intelligence (AI) and Machine Learning (ML). It serves as an essential platform ...