At a Glance
- Tasks: Build AI observability systems and monitor production models using Python and telemetry.
- Company: Join BMO, a leader in innovative banking solutions.
- Benefits: Hybrid work model, competitive salary, and opportunities for professional growth.
- Other info: Dynamic role with a focus on cutting-edge AI technologies and collaborative teamwork.
- Why this job: Make an impact by designing systems that enhance AI reliability and performance.
- Qualifications: Bachelor's degree in Computer Science or Engineering with 7-10 years of experience.
The predicted salary is between 123750 - 151250 $ per year.
Build AI observability systems at BMO in a hybrid role, using Python, telemetry, and evaluation methods for production models and agents. As a Senior AI Engineer, you will design systems that trace LLM calls, monitor token usage, detect drift, and evaluate autonomous workflows. The work includes machine learning, deep learning, statistical analysis, and specialized technical support. You will help govern agent behavior and diagnose non-routine production problems.
Key Responsibilities:
- Design and maintain AI model observability systems
- Trace agent trajectories, prompts, tools, and outputs
- Build automated evaluations for relevance and hallucinations
- Monitor drift, latency, errors, and data quality
- Define guardrails and self-remediation boundaries
Requirements:
- Bachelor's degree in Computer Science or Engineering
- Seven to ten years of Site Reliability Engineering experience
- Experience in Artificial Intelligence Observability Engineering
- Python, SQL, LogQL, or PromQL proficiency
- Experience with Prometheus, Grafana, or Datadog
Apply AI engineering, observability, and evaluation expertise to support reliable agent workflows at BMO.
Senior AI Observability Engineer in Toronto employer: Aimlroles
Procurify is an exceptional employer that fosters a collaborative and innovative work culture, allowing you to thrive as an AI Engineering Manager. With a remote-first approach, you'll enjoy the flexibility of working from anywhere in Canada while benefiting from continuous professional development opportunities and a supportive team environment that values your contributions. Join us to lead cutting-edge AI initiatives and make a meaningful impact in a company that prioritizes employee growth and well-being.
