At a Glance
- Tasks: Test and optimize AI features to enhance incident management.
- Company: Join Rootly, a leader in innovative AI solutions.
- Benefits: Competitive salary, flexible work options, and growth opportunities.
- Other info: Collaborative environment focused on innovation and quality.
- Why this job: Make a real impact on the future of AI in incident management.
- Qualifications: 5+ years in QA or AI evaluation with strong analytical skills.
The predicted salary is between 117000 - 143000 $ per year.
Join Rootly as an AI Evaluation Expert, where your insights will directly influence the future of incident management.
This role focuses on rigorous testing and optimization of AI features.
As our AI Quality Engineer, you'll play a vital role at Rootly, requiring over five years in QA or AI/ML evaluation.
Your mission is to design and execute tests, ensure quality, and collaborate closely with teams to refine AI features.
This position values individuals with sharp analytical capabilities and effective communication skills.
Key
Responsibilities
- Execute prompt-based tests covering various inputs
- Analyze AI behavior for relevance and accuracy
- Create frameworks for ongoing AI performance evaluation
- Document findings on AI behavior and improvements
- Define quality metrics for stakeholder reports
Requirements
- 5+ years in QA, AI, or related roles
- Expertise with LLM and agentic AI systems
- Excellent prompt engineering instincts
- Comfortable with evaluation tools and scripts
- Strong communicator for technical findings
Shape the future of incident management with your evaluation expertise at Rootly. #J-18808-Ljbffr
Rootly AI Evaluation Expert in Toronto employer: Rootly
At Rootly, we pride ourselves on being an exceptional employer that champions innovation and collaboration within a fully remote environment. Our commitment to employee growth is evident through our mentorship programs and focus on career development, ensuring that every team member has the opportunity to thrive. Join us in shaping the future of incident management while enjoying the flexibility and work-life balance that comes with remote work.
