Senior Software Engineer, AI in Toronto

Klue Engineering is hiring! We're looking for a Senior Machine Learning Engineer to join our team in Toronto, focusing on building and optimizing state-of-the-art LLM-powered agents that can reason, plan and automate workflows for users. You will be leading the design and development of search and retrieval agent systems that enable users to generate competitive insights for their business. In this role, you will own projects end-to-end, guiding architecture decisions, experimentation strategy, and production readiness for LLM-powered retrieval and generation workflows.

What are the responsibilities, and how will I spend my time?

• You will shape how we integrate retrieval-augmented generation (RAG), dense retrieval, query understanding, and agentic reasoning loops to deliver fast, accurate, and trusted search experiences at scale.
• Architect, design, and implement retrieval pipelines and agentic workflows, including hybrid retrieval, re-ranking, and post-retrieval synthesis.
• Lead the development of evaluation frameworks (offline and human-in-the-loop) to measure and improve relevance, quality, and latency.
• Drive experimentation with query rewriting, expansion, and classification to enhance retrieval effectiveness.
• Optimize LLM workflows by designing prompt structures, retrieval strategies, and caching for low-latency, high-accuracy responses.
• Collaborate cross-functionally with product and infrastructure teams to align technical direction with product goals.
• Mentor and provide technical guidance to team members, establishing best practices for building production-ready ML systems.
• Own data strategy for retrieval and design pipelines to automatically extract insights about competitors from both public and internal data sources.
• Evaluate and integrate advancements in LLMs, retrieval architectures, and agentic reasoning into our production systems.

What experience are we looking for?

• 5+ years of industry experience building and deploying ML systems, with at least 2+ years working on search, retrieval, or ranking systems.
• Expert-level programming skills in Python, with experience using frameworks such as PyTorch, TensorFlow, or JAX.
• Deep understanding of information retrieval (BM25, dense retrieval, hybrid retrieval) and relevance tuning.
• Experience with LLMs, retrieval-augmented generation pipelines, and prompt engineering.
• Track record of designing and delivering production-grade ML systems at scale, balancing experimentation with reliability.
• Deep understanding of data pipelines, preprocessing, and large-scale data handling.
• Familiarity with evaluation methodologies for search systems (recall, MRR, nDCG) and user-facing evaluations.
• Experience working with vector database infrastructure (FAISS, Milvus, Weaviate, Pinecone, PGVector) and traditional search engines (Elasticsearch, OpenSearch).
• Familiarity with scalable cloud ML infrastructure (AWS, GCP, Azure).
• Develop and implement CI/CD pipelines. Automate the deployment and monitoring of ML models.
• Knowledge of query understanding, document summarization, and other content enrichment strategies.
• Ability to lead projects independently while providing technical direction to others.

Nice to Have:

• Experience designing agentic LLM systems and multi-step retrieval workflows.
• Background in conversational search.
• Contributions to open-source search, retrieval, or LLM-related projects.
• Interest in publishing or sharing learnings with the broader community.

What makes you thrive at Klue?

• We're looking for builders who take ownership and run with ambiguous problems.
• Jump into new areas and rapidly learn what's needed to deliver solutions.
• Bring scientific rigor while maintaining a pragmatic delivery focus.
• See unclear requirements as an opportunity to shape the solution.

What technologies do we use?

• LLM platforms: OpenAI, Anthropic, open-source models.
• ML frameworks: PyTorch, Transformers, spaCy.
• Search/Vector DBs: Elasticsearch, Pinecone, PostgreSQL.
• MLOps tools: Weights & Biases, MLflow, Langfuse.
• Infrastructure: Docker, Kubernetes, GCP.
• Development: Python, Git, CI/CD.

What is your working style at Klue?

• Hybrid. Best of both worlds (remote & in-office). You and your team will be in the office 2 days a week.
• Our main Canadian hubs are in Vancouver and Toronto, and most of our teams are located in EST and PST.

What about Compensation & Benefits:

• Competitive base salary.
• Extended health & dental benefits that kick in Day 1.
• Opportunity to participate in our Employee Stock Option Plan.
• Take what you need time off. Just ensure the required work gets done and clear it with your team in advance.
• The average Klue team member takes 2-4 weeks of PTO per year.
• Direct access to our leadership team, including our CEO.

At Klue, we're dedicated to creating an inclusive, equitable and diverse workplace as an equal-opportunity employer. Our commitment is to build a high-performing team where people feel a strong sense of belonging, can be their authentic selves, and are able to reach their full potential. If there’s anything we can do to make our hiring process more accessible or to better support you, please let us know—we’re happy to accommodate.