Evaluate AI-generated code with Python or TypeScript expertise as an independent contractor supporting Turing's model improvement work.
This experienced software engineering role focuses on reviewing coding agents across real-world repositories. You will assess correctness, robustness, maintainability, and agent behavior, then create evaluation signals that improve coding models. The work includes collaboration with researchers and engineers.
Key Responsibilities:
- Evaluate AI-generated code across real-world software repositories
- Review agent behavior, tool usage, and code changes
- Identify technical errors and recurring model failure modes
- Create rubrics and evaluation criteria for coding tasks
- Produce preference data and actionable findings
Requirements:
- Five or more years of software engineering experience
- Strong Python, TypeScript, JavaScript, Go, or production language proficiency
- Experience with substantial real-world codebases
- Strong code-review skills and technical judgment
- Experience using modern LLMs or AI coding tools
Apply software engineering judgment, code review, and written communication to improve Turing's AI coding models.
AI Coding Model Evaluation Engineer in Ottawa employer: Turing Enterprises, Inc.
Turing Enterprises, Inc. is an exceptional employer that values innovation and efficiency, offering a fully remote work environment that promotes flexibility and work-life balance. With a strong emphasis on professional growth, employees are encouraged to enhance their skills through hands-on projects and collaborative teamwork, making it an ideal place for those looking to make a meaningful impact in data engineering. The fast-paced culture fosters creativity and quick decision-making, ensuring that your contributions lead to tangible results.
