Data Engineer in Vancouver

We are seeking a talented Data Engineer to join Amazon's Traffic Engineering organization in developing next-generation bot detection and mitigation capabilities. As a Data Engineer, you'll be at the forefront of revolutionizing how we understand and leverage Traffic data used in Bot detection and mitigation. This is a critical challenge at the intersection of ML and cybersecurity. With GenAI/LLM-powered bots now capable of mimicking human behavior, evading traditional detections, and monetizing attacks, we must revolutionize our approach from reactive to real-time protection. Your expertise will solve crucial challenges in data processing and analytics.

Imagine building the pipelines that transform raw traffic data into actionable insights. You'll tackle complex problems like real-time data processing, ensuring data accuracy across multiple platforms, and creating scalable solutions that can handle exponential growth in data volume. As the bridge between raw data and meaningful insights, you'll work with a diverse array of technologies, including Python, EMR, Spark, and Airflow, along with AWS services like Glue, Athena, and Redshift.

But what truly sets this role apart is the opportunity to collaborate with cross-functional teams – from Scientists to Software Engineers to Program/Product – ensuring that your technical solutions align perfectly with business needs. Your contribution will extend beyond just building data pipelines. You'll be a key player in maintaining data governance and compliance, implementing sophisticated access controls, and ensuring data security through encryption and audit mechanisms. This role demands not just technical expertise, but also the ability to think strategically about data architecture and its impact on business outcomes.

This role offers the unique opportunity to implement first-of-its-kind real-time behavior-based ML models that will safeguard Amazon's global infrastructure. Your work will directly impact millions of customers by ensuring the integrity and availability of Amazon's digital presence while advancing the state-of-the-art in automated threat detection and response.

Key job responsibilities:

• Drive the backbone of our data systems by building robust pipelines that turn complex data into actionable insights using AWS and Amazon's internal tools.
• Be the bridge between teams - you'll work closely with business, science, and engineering folks to turn their data needs into reality.
• Automating our infrastructure and creating smooth deployment processes.
• Play data detective by ensuring our data is clean, accurate, and trustworthy.
• Keep our data fortress secure by implementing smart governance and security measures that protect sensitive information.
• Stay ahead of the curve by optimizing our systems and bringing modern solutions to solve data challenges.
• Create user-friendly data products that help teams make better decisions through analytics and experimentation.

About the team: Our team is embarking on a journey to develop the next generation of website traffic management software that protects customers' experience. Your work will be highly visible and is instrumental in helping Amazon serve record-breaking amounts every day including flagship events such as Prime Day. If you love building high-performance software that's focused on best-in-class networking, data processing, security, and leverages AWS technologies, we have the job for you.

Basic Qualifications:

• 3+ years of data engineering experience.
• 1+ years of developing and operating large-scale data structures for business intelligence analytics using ETL/ELT processes experience.
• 1+ years of developing and operating large-scale data structures for business intelligence analytics using OLAP technologies experience.
• 1+ years of developing and operating large-scale data structures for business intelligence analytics using data modeling experience.
• 1+ years of developing and operating large-scale data structures for business intelligence analytics using SQL experience.
• 1+ years of developing and operating large-scale data structures for business intelligence analytics using Oracle experience.
• Experience with data modeling, warehousing and building ETL pipelines.

Preferred Qualifications:

• Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions.
• Experience with non-relational databases/data stores (object storage, document or key-value stores, graph databases, column-family databases).