Become a Data Pipeline Engineer leveraging Databricks, Python, and SQL for robust data solutions. This role involves close interaction with business analysts for effective data management.
In this position, your main responsibility will be to build and optimize data pipelines for high-quality data processing. Your expertise in Spark programs and data architecture will ensure reliable integration and thorough monitoring of Databricks jobs, enhancing data governance throughout the cloud environment.
Key Responsibilities:
- Build data pipelines ensuring quality and efficiency
- Develop and deploy Spark applications in Databricks
- Troubleshoot and optimize job performance
- Implement best practices in data security and management
- Review code for optimal execution and compliance
Requirements:
- 5+ years of experience in Python coding
- Extensive SQL development expertise
- Proven experience in ETL with Databricks Pyspark
- Familiarity with cloud data warehouses like Redshift
- Architectural experience in enterprise data migrations
Utilize your skills in Python and SQL as a Data Pipeline Engineer in a collaborative environment.
Data Pipeline Engineer with Databricks Focus in Mississauga employer: ALLTECH CONSULTING SVC INC
Join our dynamic team as a Solution Architect, where you'll thrive in a collaborative and innovative work culture that values your expertise in data-intensive application development. Located in a vibrant tech hub, we offer competitive benefits, continuous learning opportunities, and the chance to lead impactful projects that shape the future of our vulnerability management platforms. Here, your contributions are recognized, and you will have the support to grow both personally and professionally in a fast-paced environment.
Contact Details:
ALLTECH CONSULTING SVC INC Recruitment Team
