At a Glance
- Tasks: Lead the design and optimization of AI infrastructure for large language models in the cloud.
- Company: Fast-scaling startup redefining applied AI with a culture of innovation.
- Benefits: Competitive salary, meaningful equity, and a collaborative work environment.
- Other info: Dynamic role with significant impact on mission-critical AI systems.
- Why this job: Shape the future of AI while working with top engineers and data scientists.
- Qualifications: 6+ years in DevOps or ML infrastructure; expertise in LLMs and cloud-native architectures.
The predicted salary is between 123750 - 151250 $ per year.
Location: Downtown Toronto
Hybrid: 4 days in office
Ready to build what powers the next generation of AI? We’re looking for a Staff LLMOps Engineer to lead the design, deployment, and optimization of large language model (LLM) infrastructure on the cloud. You’ll be the driving force behind taking trained models from lab to production—scaling efficiently across multi-GPU clusters and pushing the boundaries of inference performance for enterprise-grade AI applications.
If you thrive at the intersection of AI, cloud engineering, and systems optimization, this is your chance to shape the future of large-scale model serving in a high-impact environment.
What You’ll Do
- Architect and operationalize LLM deployment pipelines on AWS and Kubernetes/EKS.
- Build and scale multi-GPU inference infrastructure for low latency, high availability, and cost efficiency.
- Optimize inference using frameworks like vLLM, SGLang, and DeepSpeed-Inference.
- Implement advanced serving techniques: continuous batching, speculative decoding, KV-cache management, and distributed scheduling.
- Collaborate with AI researchers to convert model training outputs into production-grade APIs and services.
- Establish observability and monitoring for latency, throughput, GPU utilization, and failure recovery.
- Automate provisioning, scaling, and upgrades using Terraform and CI/CD pipelines.
- Ensure compliance, security, and efficiency in multi-tenant LLM hosting for enterprise clients.
What We’re Looking For
- 6+ years in DevOps, ML infrastructure, or cloud platform engineering.
- 2+ years of direct experience deploying and optimizing LLMs or large-scale ML models.
- Expertise with GPU-accelerated inference and distributed serving environments.
- Deep familiarity with cloud-native architectures (AWS, GCP, Azure) and Kubernetes.
- Strong foundation in Python, Bash, and IaC (Terraform).
- Experience integrating monitoring tools (Prometheus, Grafana, Datadog) for performance visibility.
- Passion for building robust, scalable, and secure AI systems.
Why Join
- Lead and own mission-critical AI infrastructure at a fast-scaling startup.
- Work alongside world-class engineers, data scientists, and innovators.
- Competitive salary + meaningful equity in a company redefining applied AI.
- A culture built on innovation, technical depth, and impact—your work truly matters.
Senior LLMOps Engineer in Toronto employer: Talent To Hire Inc.
At Talent To Hire Inc., we pride ourselves on being an exceptional employer that fosters a culture of innovation and continuous improvement. As a Chief Technology Officer, you will have the opportunity to lead transformative AI initiatives in a dynamic environment, while benefiting from our commitment to employee growth and development. Located in a vibrant tech hub, we offer competitive compensation, flexible work arrangements, and a collaborative atmosphere that empowers you to make a meaningful impact.
