Trust but Verify: Ensuring Data Quality with CI/CD
In this talk, we dive into the realm of data quality assurance through Continuous Integration and Continuous Delivery (CI/CD) practices. We explore how a CI/CD pipeline acts as a safety net, catching data inconsistencies and errors before they impact the end-users. From automated testing to timely data validation, we discuss how these principles increase data reliability and trustworthiness. Revealing how we have leveraged CI/CD to create a robust data quality assurance framework.