Location: Toronto, ON, Canada
Experience: 10–14 Years
Employment Type: Contract / Full-Time
Job Summary
We are seeking an experienced Senior / Lead Azure Data Engineer with strong expertise in Microsoft Azure, Azure Data Factory, Azure Databricks, PySpark, and Azure Synapse Analytics. The ideal candidate will have hands‑on experience designing and developing scalable ETL/ELT pipelines, implementing cloud‑based data engineering solutions, and managing large‑scale data processing workloads.
Experience working with financial or banking data, data reconciliation, data quality, and enterprise data governance will be highly valued.
Key Responsibilities
- Design, develop, and implement scalable ETL/ELT pipelines using Azure Data Factory (ADF).
- Build and optimize data processing solutions using Azure Databricks, PySpark, and Spark SQL.
- Develop data ingestion frameworks integrating multiple sources into Azure Data Lake Storage Gen2.
- Design and implement cloud-based data warehouse solutions using Azure Synapse Analytics and Azure SQL.
- Develop complex SQL queries, stored procedures, and data transformation logic.
- Implement Delta Lake architecture, Delta Live Tables (DLT), and data quality validation frameworks.
- Work with Unity Catalog to implement data governance, access controls, and data lineage.
- Develop metadata-driven and parameterized pipelines for automated data processing.
- Implement batch and near-real-time data processing solutions using Azure Event Hubs and other Azure services.
- Optimize pipeline performance, Spark jobs, SQL queries, and cloud resource utilization.
- Implement secure data access using Azure Key Vault, Azure Active Directory, and managed identities.
- Develop CI/CD processes for automated deployment across development, QA, and production environments.
- Collaborate with enterprise architects, business analysts, QA teams, and stakeholders to deliver reliable data solutions.
- Ensure data accuracy, reconciliation, exception handling, and compliance with enterprise data standards.
Mandatory Technical Skills
- Strong hands‑on experience with Microsoft Azure Data Engineering.
- Azure Databricks, PySpark, and Spark SQL.
- Strong SQL Server and Azure SQL Database experience.
- ETL/ELT pipeline development and data warehousing.
- Python programming and data transformation.
- Data integration, migration, and performance optimization.
Good to Have
- Delta Lake and Delta Live Tables (DLT).
- Unity Catalog and Databricks Asset Bundles.
- Azure Functions, Logic Apps, Event Grid, and Event Hubs.
- Data governance, data quality, and reconciliation.
Experience & Qualifications
- 10+ years of experience in Data Engineering / Data Warehousing.
- 5+ years of hands‑on experience with Azure Data Engineering technologies.
- Experience designing and implementing enterprise-scale cloud data solutions.
- Strong understanding of Agile/Scrum and SDLC methodologies.
- Bachelor's degree in Computer Science, Engineering, or a related field.
Preferred Candidate Profile
A hands‑on Senior / Lead Azure Data Engineer who can independently design, develop, optimize, and support enterprise data pipelines, with strong experience in Azure Databricks, ADF, PySpark, SQL, and cloud data architecture…
#J-18808-Ljbffr
Lead Azure Data Engineer - Banking Domain in Toronto employer: Enterprise Solutions Inc.
Enterprise Solutions Inc. is an exceptional employer located in the vibrant city of Markham, Ontario, offering a dynamic work culture that fosters collaboration and innovation. With a strong emphasis on employee growth, we provide ample opportunities for professional development and skill enhancement, ensuring our team members thrive in their careers. Join us to be part of a supportive environment where your contributions are valued, and you can make a meaningful impact in the insurance technology sector.
