A senior machine learning systems engineer will optimize LLM inference performance with Python, PyTorch, and AWS Trainium remotely anywhere in Canada. The focus includes decode latency and throughput. At Capgemini Engineering, this hands-on engineer improves production-grade model serving across Trainium2 and Trainium3. The work covers vLLM on AWS Neuron, Qwen-family models, speculative decoding, multi-token prediction, and reproducible benchmarking against GPUs. You will address long-context and multi-chip scaling. Key Responsibilities:
Improve decode latency, throughput, cost, and correctness on Trainium Implement speculative decoding and native multi-token prediction Design matched tests controlling prompts, concurrency, cache, and warm-up Scale long-context serving across multi-chip Trainium systems Create runbooks, guides, documentation, and hands-on workshops Requirements:
Five-plus years in machine learning or systems engineering Two-plus years in production model inference or serving Strong Python and PyTorch proficiency modifying model implementations Experience with vLLM, TensorRT-LLM, SGLang, or Triton Understanding of prefill, decode, batching, parallelism, and cost Apply machine learning systems expertise to optimize AWS Trainium inference, benchmarking, and serving performance at Capgemini Engineering.
#J-18808-Ljbffr
Senior LLM Inference Performance Engineer in Toronto employer: Capgemini Engineering
Capgemini Engineering in Toronto is an exceptional employer that fosters a collaborative and innovative work culture, making it an ideal place for a Senior Utility Project Manager. With a strong focus on employee growth and development, the company offers numerous opportunities for professional advancement while supporting a healthy work-life balance through its hybrid work model. Employees benefit from engaging projects in the utility sector, allowing them to make a meaningful impact while building lasting client relationships.
