AI Inference Performance Engineer - New College Grad 2026 job opportunity at NVIDIA.

Home Jobs In United States Of America Tag: Job AI Inference Performance Engineer - New College Grad 2026 position at NVIDIA


DatePosted 5 Days Ago bot
NVIDIA AI Inference Performance Engineer - New College Grad 2026
Experience: 2-years
Pattern: full-time
Country:
apply Apply Now
Salary:
Status:

Job

Copy Link Report
degreePhD
loacation US, CA, Santa Clara, United States Of America
loacation US, CA, Santa ..........United States Of America

We optimize and benchmark GenAI inference on NVIDIA's latest accelerators, defining the industry’s performance standards across language models, video generation, and speech workloads. We work directly within TensorRT-LLM, SGLang, and vLLM, building the tools that evaluate serving performance at scale. This team sits at the intersection of GPU performance engineering and public accountability. What You Will Be Doing: Drive industry benchmark results: own the end-to-end optimization pipeline, implement and integrate optimizations in quantization, scheduling, memory management, and distributed inference across TensorRT-LLM, SGLang, and vLLM. Define and optimize cutting-edge workloads: identify and shape next-generation inference benchmarks, multi-turn coding, agentic workflows, and other emerging AI use cases. Collaborate with framework and kernel teams to push performance to its extreme on large-scale LLM-MoE models, vision-language models, video diffusion models, recommendation, and speech workloads. Architect distributed inference: Design and optimize execution from single-GPU to rack-scale clusters, managing performance across clusters of GPUs. Establish performance methodology: Apply roofline analysis and systematic profiling to decompose bottlenecks across CUDA kernels, frameworks, and serving layers. Influence the ecosystem: contribute to TensorRT-LLM, vLLM, SGLang, and other open-source projects. Partner with architecture, kernel, and compiler teams to shape GPU roadmaps based on real workload data. Technical Leadership: Raise the technical bar for the team, drive cross-functional execution on tight benchmark timelines, and lead a world-class team. What We Need To See: BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or equivalent experience. 2+ years of relevant software development experience. Strong Python or C++ programming, software design, and software engineering skills. Expertise with a DL framework such as PyTorch or JAX. Proven track record of delivering measurable performance improvements in deep learning inference or high-performance systems. Deep understanding of LLM/VLM architectures and inference mechanics: attention, KV caching, batching strategies, decode-phase bottlenecks, speculative decoding, disaggregated serving etc. Ways To Stand Out From The Crowd: Prior experience with an LLM framework (TensorRT-LLM, vLLM, SGLang, etc) or a DL compiler in inference, deployment, algorithms, or implementation. Prior experience with performance modeling, profiling, debug, and code optimization of a DL/HPC/high-performance application. Experience with scale-out inference orchestration (MPI, NCCL, K8S) on large GPU clusters. Expertise in kernel development (CUTLASS, cuteDSL, tilelang, OpenAI Triton) or compiler/runtime paths (torch.compile, graph lowering, operator fusion). Architectural knowledge of CPU, GPU, FPGA or other DL accelerators; GPU programming experience (CUDA). Track record of leading ambiguous, high-impact technical programs across multiple teams under tight deadlines. GPU deep learning has provided the foundation for machines to learn, perceive, reason and solve problems posed using human language. The GPU started out as the engine for simulating human imagination, conjuring up the outstanding virtual worlds of video games and Hollywood films. Now, NVIDIA's GPU runs deep learning algorithms, simulating human intelligence, and acts as the brain of computers, robots and self-driving cars that can perceive and understand the world. Just as human imagination and intelligence are linked, computer graphics and artificial intelligence come together in our architecture. Two modes of the human brain, two modes of the GPU. This may explain why NVIDIA GPUs are used broadly for deep learning, and NVIDIA is increasingly known as “the AI computing company.” Come, join our DL Architecture team, where you can help build the real-time, cost-effective computing platform driving our success in this exciting and quickly growing field. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 124,000 USD - 195,500 USD for Level 2, and 152,000 USD - 241,500 USD for Level 3. You will also be eligible for equity and benefits . Applications for this job will be accepted at least until March 9, 2026. This posting is for an existing vacancy.  NVIDIA uses AI tools in its recruiting processes. NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Other Ai Matches

Senior ASIC Design Engineer, Memory Controller Applicants are expected to have a solid experience in handling Memory Controller related tasks
Implementation Methodology Engineer - GPU Applicants are expected to have a solid experience in handling Job related tasks
CPO Integration Engineer - LPU Packaging Applicants are expected to have a solid experience in handling Job related tasks
Principal Quantum Error Correction Research Scientist, Applied Research Applicants are expected to have a solid experience in handling Applied Research related tasks
System Software Engineer - Tegra Applicants are expected to have a solid experience in handling Job related tasks
AI Compiler Engineer Applicants are expected to have a solid experience in handling Job related tasks
Senior Software Developer Applicants are expected to have a solid experience in handling Job related tasks
System Design Engineer Applicants are expected to have a solid experience in handling Job related tasks
Senior GenAI Engagement Lead, Partner Platforms Applicants are expected to have a solid experience in handling Partner Platforms related tasks
Senior Product Quality and Reliability Engineer Applicants are expected to have a solid experience in handling Job related tasks
Manager, MCU Firmware Applicants are expected to have a solid experience in handling MCU Firmware related tasks
Formal Verification Infra Development Engineer Applicants are expected to have a solid experience in handling Job related tasks
remote-jobserver Remote
Manager, Software Verification Applicants are expected to have a solid experience in handling Software Verification related tasks
Director, Technical Accounting Applicants are expected to have a solid experience in handling Technical Accounting related tasks
Senior PR Manager, AI Software Applicants are expected to have a solid experience in handling AI Software related tasks
Senior ASIC Engineer - Networking Chip Design Applicants are expected to have a solid experience in handling Job related tasks
Sales Development Specialist Applicants are expected to have a solid experience in handling Job related tasks
Senior System Software Engineer, Networking Linux Kernel - DGX Cloud Applicants are expected to have a solid experience in handling Networking Linux Kernel - DGX Cloud related tasks
Senior Technical Program Manager, Metropolis Manufacturing Applicants are expected to have a solid experience in handling Metropolis Manufacturing related tasks
Manager, Large Language Model Inference Applicants are expected to have a solid experience in handling Large Language Model Inference related tasks
Tools and Development Engineer Applicants are expected to have a solid experience in handling Job related tasks
System Architect Applicants are expected to have a solid experience in handling Job related tasks
Senior ASIC Verification Engineer Applicants are expected to have a solid experience in handling Job related tasks