We are seeking a Senior Software Quality Engineer – AI Focused to drive quality, reliability, and automation for AI-enabled software products. The role will collaborate with Development, DevOps, Product, and QE teams to design robust testing strategies and scalable automation frameworks. The engineer will test AI/ML and Generative AI features, APIs, microservices, integrations, and data pipelines while ensuring reliable and secure product delivery. The position is full-time and onsite in Toronto, ON.
Key Requirements
- 5+ years of experience in Quality Engineering, QA, SDET, or Test Automation.
- 5+ years of programming experience with Java, Python, Groovy, or similar languages.
- Strong experience building and maintaining test automation frameworks.
- Hands-on experience with Selenium, Playwright, Cucumber/Gherkin, or RestAssured.
- Experience with API, database, integration, regression, and microservices testing.
- Experience with JUnit, TestNG, ReadyAPI, qTest, and JMeter.
- Hands-on experience testing AI/ML, Generative AI, conversational AI, or LLM-based applications.
- Knowledge of AI quality concepts such as prompt evaluation, accuracy, relevance, grounding, hallucination detection, toxicity, bias, and response consistency.
- Experience with CI/CD, Agile, BDD, TDD, ATDD, and risk-based testing.
- Strong analytical, problem-solving, communication, and documentation skills.
Key Responsibilities
- Develop and execute comprehensive quality strategies, test plans, test cases, and test data.
- Design, develop, and maintain scalable test automation frameworks.
- Create automated tests for AI/ML and Generative AI features.
- Evaluate AI responses for accuracy, relevance, safety, hallucination, reliability, and latency.
- Test APIs, microservices, databases, integrations, data pipelines, and third-party services.
- Create AI evaluation datasets, expected behaviors, quality thresholds, and regression coverage.
- Integrate automated testing and AI quality checks into CI/CD pipelines.
- Identify, document, prioritize, and track defects through resolution.
- Analyze test coverage, quality metrics, AI evaluation results, and production feedback.
- Collaborate with Development, DevOps, Product, and QE teams to improve product quality and release readiness.
#J-18808-Ljbffr
Senior Software Quality Engineer – AI in Toronto employer: PeoplePilot
Join a forward-thinking company that prioritizes innovation and collaboration in the realm of Governance, Risk, and Compliance. As a Senior Archer Solution Architect, you will thrive in a dynamic work culture that values employee growth and offers extensive professional development opportunities. Located in a vibrant city, our organization provides a supportive environment where your expertise will directly impact compliance initiatives and drive meaningful change.
