Self-Supervised Learning Strategies for Reducing Label Dependency in Large-Scale Data Science Applications

Kit Ling Chan (Hong Kong Shue Yan University)
Mohd Zaki Zakaria (University Technology Mara)
Eka Puji Agustini (Universitas Bina Darma)

International Journal of Advances in Artificial Intelligence and Machine Learning
Vol. 3 No. 2 (2026): International Journal of Advances in Artificial Intelligence and Machine Learni

Background: The rapid expansion of large-scale data ecosystems has intensified the challenge of acquiring high-quality labeled datasets for supervised learning models. Although raw data availability continues to grow exponentially, manual annotation remains costly, time-consuming, and susceptible to inconsistency, creating a structural bottleneck in scalable machine learning deployment. Existing self-supervised learning (SSL) approaches have demonstrated promising results in domain-specific applications, particularly in computer vision. However, many lack a unified, modality-agnostic framework and systematic evaluation under realistic constraints such as limited-label availability and noisy annotations.Aims: This study aims to develop a unified self-supervised learning framework that reduces label dependency across heterogeneous data modalities. The scope of the research includes image and tabular datasets, focusing on evaluating model robustness under low-label conditions and label noise scenarios.Methods: The proposed framework integrates contrastive representation learning with reconstruction-based objectives in a two-stage training strategy. First, large-scale unlabeled pre-training is conducted to learn robust feature representations. This is followed by supervised fine-tuning under controlled label fractions (1%–20%) and synthetic label corruption levels of up to 30%. Performance is evaluated against fully supervised baselines using accuracy and data efficiency metrics.Result: Experimental findings demonstrate that the proposed SSL framework achieves up to 15% higher accuracy than fully supervised baselines under 1% labeled data conditions. Moreover, it maintains approximately 85–90% of clean-label performance even with 30% label corruption. The data efficiency ratio improves by up to 2.3× in low-label regimes, indicating substantial gains in learning effectiveness under constrained supervision.Conclusion: The results confirm that self-supervised pre-training significantly enhances representation quality, robustness, and scalability while reducing annotation requirements. This research establishes SSL as a foundational paradigm for developing data-efficient and resilient AI systems in large-scale analytics environments.