Design generative AI evaluation systems for EA-SPORTS FC at Electronic Arts, combining LLM-as-judge patterns, metrics, and automated testing. Work hybrid in Burnaby, BC. This Software Engineer III role sits within FC Quality Verification Engineering. You will create evaluation frameworks and developer tools that identify quality issues earlier. Your work spans dataset curation, model evaluation, validation criteria, error analysis, and deterministic checks. Key Responsibilities:
Develop evaluation frameworks for generative AI systems Implement LLM-as-judge patterns and rubric-based scoring Perform automated calibration alongside deterministic validation checks Define methods for exploring complex software behaviour Produce reproducible diagnostics and support delivered software Requirements:
Experience with generative system evaluation frameworks Knowledge of LLM architectures, concepts, and failure modes Experience curating datasets and defining validation criteria Experience with quantitative and qualitative evaluation metrics Master of Science or equivalent engineering experience Bring generative AI evaluation, LLM analysis, and software testing expertise to Electronic Arts’ EA-SPORTS FC engineering work.
#J-18808-Ljbffr
Generative AI Evaluation Engineer in Vancouver employer: Electronic Arts (EA)
At Electronic Arts, we pride ourselves on fostering a dynamic and innovative work culture that empowers our employees to push the boundaries of gaming technology. As a Technical Director in our vibrant studio, you will have access to cutting-edge resources and collaborative teams, ensuring your professional growth while contributing to groundbreaking projects. Located in a hub of creativity and technology, we offer competitive benefits and a supportive environment that values diversity and encourages continuous learning.
