Understanding the Capabilities of Data Quality Measurement and Monitoring

Normi Sham Awang Abu Bakar (International Islamic University Malaysia)
Alea Syaffa Mohd Sabri (International Islamic University Malaysia)
Aisya Sufiah Hazlan (International Islamic University Malaysia)

Journal of Sustainable Software Engineering and Information Systems
Vol. 1 No. 1 (2025): Journal of Sustainable Software Engineering and Information Systems

Background of Study: Ensuring high-quality data is essential for organizations that depend on analytics, automation, and regulatory compliance. Aims and Scope of Paper: This paper explores the foundational concepts and evolving practices of two interrelated capabilities: data quality measurement and data quality monitoring. While measurement focuses on quantifying attributes such as accuracy, completeness, consistency, and timeliness, monitoring emphasizes the continuous detection and alerting of anomalies during data operations. Methods: This paper examines the application of frameworks like Total Data Quality Management (TDQM), ISO 8000, and Data Management Association Data Management Body of Knowledge (DAMA DMBOK), alongside emerging tools such as rule-based engines, metadata-driven platforms, and AI-driven anomaly detection systems. Results: Findings reveal a persistent gap in systems that integrate both measurement and monitoring effectively, hindering long-term data governance. This paper discusses a case study of the Data Quality framework implementation in the Healthcare sector. It was found that the healthcare organization implemented the Total Data Quality Management (TDQM) framework and Apache Griffin to ensure the accuracy, completeness, consistency, timeliness, and validity of clinical and IoT data through continuous monitoring, automated validation, and anomaly detection. Governance mechanisms aligned with ISO 8000 and HIPAA standards ensured full compliance, traceability, and accountability across all data quality and auditing processes. This study contributes to a deeper understanding of how integrated data quality practices can support digital transformation and operational resilience across industries. Conclusion: The paper concludes by recommending the adoption of continuous quality measurement practices aligned with governance policies and supported by both human expertise and automation, arguing that data quality must be embedded as a dynamic and strategic function within the digital enterprise. While measurement emphasizes the quantification of data attributes such as accuracy, completeness, consistency, and timeliness, monitoring focuses on the continuous detection and alerting of data anomalies during data operations.