Optimize large language model inference with Python and PyTorch across AWS Trainium platforms, including Trainium2 and Trainium3. Improve production AI performance through hands-on accelerator optimization. As a Senior Machine Learning Systems Engineer, you will improve inference latency, throughput, scalability, and serving costs. The work includes LLM inference frameworks, AWS Neuron SDK, vLLM on Neuron, and production workload optimization. You will evaluate advanced techniques for Qwen-family models. Key Responsibilities:
Optimize decode latency, throughput, and LLM serving costs Work with Trainium2, Trainium3, Neuron SDK, and vLLM Evaluate speculative decoding and Multi-Token Prediction for Qwen models Profile and benchmark GPU and Trainium production workloads Optimize prefill, decode, batching, KV-cache, and parallelism Requirements:
Five-plus years in Machine Learning or Systems Engineering Two-plus years with production LLM inference or serving Strong Python and PyTorch programming experience Experience with vLLM, TensorRT-LLM, SGLang, or Triton Understanding of prefill, decode, batching, and KV-cache management Apply inference performance engineering, Python, PyTorch, and accelerator optimization across demanding production LLM workloads.
#J-18808-Ljbffr
Senior LLM Inference Performance Engineer in Winnipeg employer: Aptino, Inc.
Aptino, Inc. is an exceptional employer located in the vibrant city of Toronto, offering a dynamic work culture that fosters innovation and collaboration. As a Senior Salesforce Technical Lead, you will not only have the opportunity to work on cutting-edge projects for a major Canadian banking client but also benefit from extensive employee growth opportunities, mentorship programs, and a commitment to work-life balance. Join us to be part of a forward-thinking team that values your expertise and encourages professional development in a supportive environment.
