Senior Software Engineer, AI Inference Systems job opportunity at NVIDIA.

Home Jobs In Canada Tag: AI Inference Systems Senior Software Engineer, AI Inference Systems position at NVIDIA


DatePosted 30+ Days Ago bot
NVIDIA Senior Software Engineer, AI Inference Systems
Experience: 7-years
Pattern: full-time
Country:
apply Apply Now
Salary:
Status:

AI Inference Systems

Copy Link Report
degreeOND
loacation Canada, Toronto, Canada
loacation Canada, Toront..........Canada

We are seeking highly skilled and motivated software engineers to join us and build AI inference systems that serve large-scale models with extreme efficiency. You’ll architect and implement high-performance inference stacks, optimize GPU kernels and compilers, drive industry benchmarks, and scale workloads across multi-GPU, multi-node, and multi-cloud environments. You’ll collaborate across inference, compiler, scheduling, and performance teams to push the frontier of accelerated computing for AI. What you’ll be doing: Contribute features to vLLM that empower the newest models with the latest NVIDIA GPU hardware features; profile and optimize the inference framework (vLLM) with methods like speculative decoding, data/tensor/expert/pipeline-parallelism, prefill-decode disaggregation. Develop, optimize, and benchmark GPU kernels (hand-tuned and compiler-generated) using techniques such as fusion, autotuning, and memory/layout optimization; build and extend high-level DSLs and compiler infrastructure to boost kernel developer productivity while approaching peak hardware utilization. Define and build inference benchmarking methodologies and tools; contribute both new benchmark and NVIDIA’s submissions to the industry-leading MLPerf Inference benchmarking suite. Architect the scheduling and orchestration of containerized large-scale inference deployments on GPU clusters across clouds. Conduct and publish original research that pushes the pareto frontier for the field of ML Systems; survey recent publications and find a way to integrate research ideas and prototypes into NVIDIA’s software products. What we need to see: Bachelor’s degree (or equivalent expeience) in Computer Science (CS), Computer Engineering (CE) or Software Engineering (SE) with 7+ years of experience; alternatively, Master’s degree in CS/CE/SE with 5+ years of experience; or PhD degree with the thesis and top-tier publications in ML Systems, GPU architecture, or high-performance computing. Strong programming skills in Python and C/C++; experience with Go or Rust is a plus; solid CS fundamentals: algorithms & data structures, operating systems, computer architecture, parallel programming, distributed systems, deep learning theories. Knowledgeable and passionate about performance engineering in ML frameworks (e.g., PyTorch) and inference engines (e.g., vLLM and SGLang). Familiarity with GPU programming and performance: CUDA, memory hierarchy, streams, NCCL; proficiency with profiling/debug tools (e.g., Nsight Systems/Compute). Experience with containers and orchestration (Docker, Kubernetes, Slurm); familiarity with Linux namespaces and cgroups. Excellent debugging, problem-solving, and communication skills; ability to excel in a fast-paced, multi-functional setting. Ways to stand out from the crowd Experience building and optimizing LLM inference engines (e.g., vLLM, SGLang). Hands-on work with ML compilers and DSLs (e.g., Triton, TorchDynamo/Inductor, MLIR/LLVM, XLA), GPU libraries (e.g., CUTLASS) and features (e.g., CUDA Graph, Tensor Cores). Experience contributing to containerization/virtualization technologies such as containerd/CRI-O/CRIU. Experience with cloud platforms (AWS/GCP/Azure), infrastructure as code, CI/CD, and production observability. Contributions to open-source projects and/or publications; please include links to GitHub pull requests, published papers and artifacts. At NVIDIA, we believe artificial intelligence (AI) will fundamentally transform how people live and work. Our mission is to advance AI research and development to create groundbreaking technologies that enable anyone to harness the power of AI and benefit from its potential. Our team consists of experts in AI, systems and performance optimization. Our leadership includes world-renowned experts in AI systems who have received multiple academic and industry research awards. If you’re excited to build systems, kernels, and tools that make large-scale AI faster, more efficient, and easier to deploy, we’d love to hear from you. #LI-Hybrid Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 170,000 CAD - 220,000 CAD for Level 4, and 225,000 CAD - 275,000 CAD for Level 5. You will also be eligible for equity and benefits . Applications for this job will be accepted at least until January 27, 2026. This posting is for an existing vacancy.  NVIDIA uses AI tools in its recruiting processes.

Other Ai Matches

remote-jobserver Remote
Senior Software Architect - Deep Learning and HPC Communications Applicants are expected to have a solid experience in handling Job related tasks
Senior System Software Engineer - Tegra MODS Team Applicants are expected to have a solid experience in handling Job related tasks
Senior Product Development Engineer Applicants are expected to have a solid experience in handling Job related tasks
Senior Product Margin Data Analyst Applicants are expected to have a solid experience in handling Job related tasks
remote-jobserver Remote
Senior Solutions Architect, Public Sector Applicants are expected to have a solid experience in handling Public Sector related tasks
Senior DFX Methodology Engineer Applicants are expected to have a solid experience in handling Job related tasks
remote-jobserver Remote
Inception Regional Lead, DACH Applicants are expected to have a solid experience in handling DACH related tasks
remote-jobserver Remote
Solution Architect - Generative AI and Post-Training Applicants are expected to have a solid experience in handling Job related tasks
Senior Developer Relations Manager Applicants are expected to have a solid experience in handling Job related tasks
Data Center Network Deployment Engineer Applicants are expected to have a solid experience in handling Job related tasks
Manager, Deep Learning Algorithms Applicants are expected to have a solid experience in handling Deep Learning Algorithms related tasks
Senior Research Scientist, Multi-Modal Language Models Applicants are expected to have a solid experience in handling Multi-Modal Language Models related tasks
Senior Financial Analyst Applicants are expected to have a solid experience in handling Job related tasks
PCB Design Layout Engineer Applicants are expected to have a solid experience in handling Job related tasks
Senior GPU Compiler Development Engineer Applicants are expected to have a solid experience in handling Job related tasks
Senior Hardware Time Synchronization Architect Applicants are expected to have a solid experience in handling Job related tasks
Post Silicon Hardware System Integration Engineer' Applicants are expected to have a solid experience in handling Job related tasks
Senior Architect- Molecular Dynamics Applicants are expected to have a solid experience in handling Job related tasks
Senior Systems Software Engineer, Cloud Infrastructure and Development Applicants are expected to have a solid experience in handling Cloud Infrastructure and Development related tasks
remote-jobserver Remote
Product Marketing Manager, Quantum Computing Platform Applicants are expected to have a solid experience in handling Quantum Computing Platform related tasks
remote-jobserver Remote
Director, Global AI Initiatives - EMEA Applicants are expected to have a solid experience in handling Global AI Initiatives - EMEA related tasks
Principal Datacenter Resiliency Architect, RAS Features and Modeling Applicants are expected to have a solid experience in handling RAS Features and Modeling related tasks
Senior Compiler Engineer - Compute Front-End Applicants are expected to have a solid experience in handling Job related tasks