At a Glance
- Tasks: Develop and maintain Azure Data solutions with a focus on Databricks and data governance.
- Company: Join a leading financial firm known for innovation and teamwork.
- Benefits: Competitive salary, flexible work options, and opportunities for professional growth.
- Other info: Dynamic team environment with a focus on data security and compliance.
- Why this job: Make an impact in the financial sector while working with cutting-edge technology.
- Qualifications: 7+ years in Azure Data Factory, Databricks, and strong Python skills.
The predicted salary is between 123750 - 151250 $ per year.
Working with one of our top financial clients, this role calls for a Sr. Azure Data Engineer-DataBricks. This position involves working within a dynamic team to support large-scale data solutions in the asset management sector, focusing on advanced Databricks implementations, data governance, and cloud-based data platforms to enable strategic decision-making.
Responsibilities
- Develop and maintain Azure Data Factory orchestrations with Databricks integrations
- Configure and manage Unity Catalog for data governance and access control
- Design and implement Delta Lake schemas and tables with appropriate security permissions
- Load datasets from diverse sources into Databricks using ADF with Medallion architecture
- Convert Databricks workspace from hive metastore to Unity Catalog and set up associated permissions
- Build and optimize Delta Live Tables, ensuring performance and cost-efficiency
- Develop deployment pipelines using Azure DevOps and GitHub Enterprise for CI/CD processes
- Deploy and manage Azure PaaS resources such as Azure SQL, Azure Data Lake, and Synapse
- Collaborate with team members independently and provide technical leadership
- Support a highly regulated environment with a focus on data security, governance, and compliance
- Troubleshoot and resolve complex data platform issues efficiently
Desired Skill-Set
- 7+ years' experience with Azure Data Factory, Databricks, and Azure Functions (C#)
- Deep knowledge of Python, notebooks, Spark, and pandas within Databricks
- Expertise in Unity Catalog and data governance setup in a cloud environment
- Experience loading datasets from various sources using ADF in a Medallion architecture
- Proficiency in converting and managing Databricks workspace permissions and security
- Familiarity with Delta Live Tables, cost, and performance optimization
- Experience with Azure DevOps or GitHub for DevOps pipelines and CI/CD
- Working knowledge of Azure Entra ID for SSO and RBAC configuration
- Ability to work in a highly regulated industry environment with limited access
- Strong communication skills and ability to lead small teams independently
- Financial services industry experience is an asset
Nice to Have
- Previous experience working with financial institutions
Sr. Azure Data Engineer in Toronto employer: BeachHead Inc.
At BeachHead, we pride ourselves on fostering a dynamic and inclusive work environment that empowers our employees to thrive. As a Senior Swaps Business Analyst, you will have the opportunity to collaborate with top-tier professionals in the financial sector, driving innovation in electronic trading and enhancing trading workflows. Our commitment to employee growth is reflected in our supportive culture, where continuous learning and development are encouraged, making us an exceptional employer for those seeking meaningful and rewarding careers in the heart of capital markets.
