At a Glance
- Tasks: Design and build scalable data pipelines using Spark, Kafka, and Flink.
- Company: Join Index Exchange, a leader in data engineering innovation.
- Benefits: Competitive salary, flexible work options, and opportunities for growth.
- Other info: Collaborative environment with a focus on leadership and innovation.
- Why this job: Make an impact by transforming high-volume data into actionable insights.
- Qualifications: Strong data engineering experience and expertise in distributed processing.
The predicted salary is between 121500 - 148500 $ per year.
Design batch and streaming pipelines at Index Exchange using Spark, Kafka, and Flink for billions of daily events. Build reliable data systems at scale. As a Staff Data Engineer, you will evolve the data processing and pipeline layer supporting reporting, billing, and real-time data streams. You will transform high-volume event data into clean, queryable, production-grade datasets. The work spans distributed processing, SQL, data modeling, workflow orchestration, and performance optimization.
Key Responsibilities:
- Design and operate Spark, Kafka, and Flink pipelines
- Build canonical datasets and reporting tables
- Maintain Airflow DAGs for scheduling and backfills
- Optimize runtime, throughput, compute efficiency, and scalability
- Define data processing standards across teams
Requirements:
- Strong experience in data engineering at scale
- Expertise in distributed processing frameworks, SQL, and data modeling
- Experience with Airflow, Kafka, or streaming systems
- Experience managing Kubernetes
- Ability to lead projects and debug complex data issues
- Bring staff-level data engineering leadership to Index Exchange through scalable pipelines, streaming systems, and optimized data platforms.
Staff Data Engineer Pipelines in Toronto employer: Index Exchange
At Index Exchange, we pride ourselves on being an exceptional employer that fosters a culture of innovation and collaboration. Our comprehensive benefits package, including health plans, flexible work schedules, and generous parental leave, ensures that our employees feel valued and supported. With a strong emphasis on professional development and a commitment to diversity and inclusion, we empower our team members to grow and thrive in their careers while making a meaningful impact in the digital advertising landscape.
