Become a key player as a Data Quality Tester, ensuring data integrity within Azure and Databricks environments. Your focus will be on validating data pipelines and developing automation scripts for data processing quality assurance.
This role encompasses the development and execution of test strategies, alongside the validation of ETL/ELT pipelines. You'll perform crucial data validation and monitoring across Delta tables while integrating testing procedures into CI/CD workflows to enhance overall project efficiency.
Key Responsibilities:
- Create and implement test strategies for data pipelines
- Validate ETL/ELT pipelines in Azure Data Factory
- Carry out detailed data validation and reconciliation
- Monitor quality across Delta tables and data warehouses
- Collaborate with teams for defect documentation and resolution
Requirements:
- Profound knowledge of Azure Data Factory and Databricks
- Strong SQL skills; Python is preferred
- Experience with data profiling and schema validation
- Familiarity with tools like Azure DevOps and GitHub Actions
- Understanding SCD Type 2 and transformation logic
Leverage your testing skills to enhance data processes in Azure and Databricks.
Data Quality Tester for Data Pipelines in Alton employer: Compunnel Inc.
Join a forward-thinking company in Toronto that prioritizes innovation and employee development. As a Digital Banking Engineering Manager, you'll thrive in a collaborative work culture that values continuous improvement and offers ample opportunities for professional growth. With a focus on modern banking solutions, this role not only allows you to lead a talented team but also empowers you to make a meaningful impact in the digital banking landscape.
