At a Glance
- Tasks: Validate data pipelines and ensure data integrity using Azure and Databricks.
- Company: Join a forward-thinking tech company focused on data excellence.
- Benefits: Competitive pay, flexible hours, and opportunities for skill development.
- Other info: Collaborative environment with great potential for career advancement.
- Why this job: Make a real impact by ensuring high data quality in innovative projects.
- Qualifications: Knowledge of Azure Data Factory, Databricks, SQL, and scripting languages.
The predicted salary is between 82800 - 101200 $ per year.
We are looking for a Data Quality Tester to validate data pipelines and ensure data integrity across Azure and Databricks environments.
This role involves developing test strategies, automation scripts, and collaborating with engineering teams to maintain high data quality standards.
Key Responsibilities
- Develop and execute test strategies and automation scripts for data pipelines.
- Validate ETL/ELT pipelines built using Azure Data Factory and Databricks.
- Perform data validation, reconciliation, and comparative analysis between source and target systems.
- Monitor data quality across Delta tables and data warehouses.
- Integrate data testing into CI/CD workflows.
- Document defects and collaborate with engineering teams for resolution.
- Required Qualifications
- Strong knowledge of Azure Data Factory and Databricks.
- Proficiency in SQL and scripting languages (Python preferred).
- Experience with data profiling, reconciliation, and schema validation.
- Familiarity with Dev Ops tools like Azure Dev Ops or Git Hub Actions.
- Understanding of SCD Type 2 and transformation logic.
Email ID * This field is required Please enter valid email Id. Cell phone * This field is required Please enter valid cell phone.
First Name * This field is required Please enter valid first name.
Last Name * This field is required Please enter valid last name.
#J-18808-Ljbffr
Data Quality Tester for Azure Data Factory & Databricks employer: Compunnel Inc.
Join a forward-thinking company in Toronto that prioritizes innovation and employee development. As a Digital Banking Engineering Manager, you'll thrive in a collaborative work culture that values continuous improvement and offers ample opportunities for professional growth. With a focus on modern banking solutions, this role not only allows you to lead a talented team but also empowers you to make a meaningful impact in the digital banking landscape.
