At a Glance
- Tasks: Design and develop scalable cloud data platforms and pipelines using Azure technologies.
- Company: Join a leading tech firm focused on innovative cloud solutions.
- Benefits: Attractive salary, health benefits, remote work options, and growth opportunities.
- Other info: Dynamic team environment with opportunities for career advancement.
- Why this job: Make an impact in the cloud data space with cutting-edge tools and technologies.
- Qualifications: Strong Python and SQL skills, plus experience with Databricks and ETL pipelines.
The predicted salary is between 123750 - 151250 $ per year.
The Azure Cloud Data Engineer is responsible for designing, developing, and maintaining scalable cloud-based data platforms and data pipelines within Azure environments.
This role focuses on building high-performance ETL pipelines, developing Databricks and Py Spark solutions, optimizing large-scale data processing, and implementing enterprise data warehouse architectures.
The ideal candidate will have strong expertise in Python, SQL, Azure Databricks, cloud data engineering, and enterprise data platform modernization.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines for data ingestion, transformation, and loading from multiple data sources.
- Build and deploy Spark applications using Azure Databricks and Py Spark to process large-scale datasets.
- Optimize Databricks jobs for performance, scalability, and reliability.
- Monitor, troubleshoot, and resolve issues impacting data pipelines and Databricks workloads.
- Design and implement enterprise data warehouse solutions and dimensional data models.
- Develop solutions utilizing Delta Lake, cloud data integration, and modern data architecture principles.
- Develop and maintain applications using Python and SQL.
- Create and optimize complex SQL queries, stored procedures, and reverse engineer existing database processes.
- Design and implement data models supporting OLTP, OLAP, dimensional modeling, facts, and dimensions.
- Process structured, semi-structured, and unstructured data.
- Develop event-driven and streaming data ingestion solutions.
- Implement best practices for data governance, security, quality, and cloud data management.
- Perform code reviews to ensure compliance with development standards and optimal execution patterns.
- Support enterprise cloud data platform migrations and modernization initiatives.
- Collaborate with cross-functional teams to design scalable cloud-based data architectures.
- Required Qualifications
- Bachelor's degree in Computer Science, Engineering, or a related technical field.
- 9+ years of overall data engineering experience.
- 5+ years of hands-on Python development experience.
- 5+ years of SQL Server development experience working with large datasets.
- 5+ years of experience developing and deploying ETL pipelines using Databricks and Py Spark.
- Experience with Delta Lake, data integration, cloud architecture, and data modeling.
- Experience with enterprise data warehouse design and dimensional modeling.
- Experience processing structured, semi-structured, and unstructured data.
- Experience with event-driven and streaming data processing technologies.
Experience with cloud data warehouse platforms such as Azure Synapse, Snowflake, Amazon Redshift, or Google Big Query.
- Strong understanding of OLTP, OLAP, facts, dimensions, and enterprise data warehousing concepts.
- Experience designing and implementing enterprise cloud data architectures.
- Experience leading enterprise cloud data platform migration initiatives.
- Experience working with banking or financial services clients.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent written and verbal communication skills.
- Preferred Qualifications
- Master's degree in Computer Science, Engineering, or a related technical field.
- Experience with Apache Airflow.
- Experience with cloud-based messaging and analytics platforms.
- #J-18808-Ljbffr
Azure Cloud Data Engineer in Mississauga employer: Compunnel Inc.
Join a forward-thinking company in Toronto that prioritizes innovation and employee development. As a Digital Banking Engineering Manager, you'll thrive in a collaborative work culture that values continuous improvement and offers ample opportunities for professional growth. With a focus on modern banking solutions, this role not only allows you to lead a talented team but also empowers you to make a meaningful impact in the digital banking landscape.
