Senior AI/ML Quality Assurance Engineer in Toronto

Company Overview

If you are passionate about water and technology, Trinnex is the place for you! Trinnex is a visionary company that is transforming the way water resources are managed and protected. By combining cutting-edge digital technologies, such as sensor/IoT data, models, geospatial data, and AI/machine learning, we create innovative, smart, and scalable solutions that make a difference. Whether it's optimizing water supply and demand, detecting leaks and anomalies, or enhancing water quality and resilience, Trinnex delivers value and impact to public sector clients across the country.

Job Description

At this time, we are only able to consider candidates who reside and work in British Columbia, Alberta, Manitoba, or Ontario, where the company is registered to operate. Trinnex, wholly owned subsidiary of CDM Smith, is seeking a highly skilled and collaborative Senior AI/ML Quality Assurance Engineer to lead the validation, testing, and monitoring of production AI and machine learning solutions. This role is responsible for ensuring our predictive models are accurate, reliable, explainable, and compliant, while partnering with Data Science, MLOps, Consulting, and Governance teams to support responsible AI delivery. The ideal candidate combines expertise in AI/ML testing, model performance monitoring, and automation with the ability to translate complex technical findings into clear, actionable insights for both technical and non-technical stakeholders. This position will play a critical role in maintaining the quality and trustworthiness of Trinnex's AI-powered products across infrastructure, environmental compliance, and operational monitoring solutions.

Key Responsibilities

• Design, develop, and maintain automated testing frameworks to validate machine learning models, data pipelines, and predictive analytics solutions.
• Validate tabular, geospatial, classification, and risk-scoring models to ensure accuracy, reliability, and alignment with business and regulatory requirements.
• Perform model explainability and quality assessments using industry-standard XAI techniques and document results for internal and client-facing stakeholders.
• Partner with MLOps teams to implement monitoring, alerting, and performance tracking for data drift, concept drift, and model degradation in production environments.
• Support root cause analysis, incident investigations, rollback validation, and recovery testing for AI/ML systems.
• Assist in evaluating emerging Generative AI and Retrieval-Augmented Generation (RAG) solutions, including model performance, retrieval accuracy, safety, and guardrail effectiveness.
• Collaborate with Consulting and Governance teams to develop validation reports, model documentation, and compliance artifacts that support transparent and responsible AI practices.
• Conduct quality reviews of AI and data science deliverables prior to client release, identifying risks, data quality concerns, and model limitations.

Skills & Abilities

• Experience validating, testing, or monitoring production machine learning models, particularly classification, regression, or predictive analytics solutions.
• Strong proficiency in Python, SQL, and Git, with hands-on experience handling spatial data structures (e.g., GeoPandas, Shapely, or PostGIS).
• Experience with AI/ML testing, validation, and monitoring frameworks, including automated testing, model performance monitoring, and observability solutions (e.g. pytest, MLflow, or Deepchecks; Evidently AI, Arize, or cloud observability stacks like GCP Production Monitoring/Cloud Monitoring, Grafana, or Datadog).
• Familiarity with Docker, Kubernetes, and cloud environments (GCP), including application monitoring and production support workflows.
• Ability to translate complex data science and model outputs into clear, actionable documentation for technical and business stakeholders.
• Understanding of model performance measurement, drift detection, and production monitoring concepts.
• Strong technical writing skills with experience creating and reviewing model documentation, validation reports, test plans, runbooks, APIs, or other technical artifacts.
• Ability to collaborate across multidisciplinary teams, including Data Science, MLOps, Software Engineering, Product Management, and Consulting organizations.

Qualifications

• Bachelor's degree.
• 5 years of related experience. Equivalent additional directly related experience will be considered in lieu of a college degree.
• Domestic and/or international travel may be required. The frequency of travel is contingent on specific duties, responsibilities, and the essential functions of the position, which may vary depending on workload and project demands.

Preferred Qualifications

• Bachelor's degree in Computer Science, Software Engineering, Data Science, Geographic Information Systems (GIS), Engineering, or a related quantitative field.
• Experience validating machine learning models, datasets, or analytical solutions within infrastructure, utilities, environmental compliance, water, or wastewater domains.
• Familiarity with geospatial data, spatial analytics, or GIS-based applications.
• Experience working with time-series, sensor, IoT, or SCADA data in operational environments.
• Knowledge of AI governance, model risk management, or responsible AI frameworks.
• Familiarity with cloud-based AI/ML platforms and production monitoring environments.
• Experience evaluating or testing Generative AI, large language model (LLM), or Retrieval-Augmented Generation (RAG) solutions.
• Understanding model explainability, fairness, bias assessment, and AI transparency concepts.

Amount Of Travel Required

No Travel is required.

Additional Information

This job posting is for an existing vacancy.