Principal Software Engineer - Inference as a Service job opportunity at NVIDIA.

Home Jobs In United States Of America Tag: Job Principal Software Engineer - Inference as a Service position at NVIDIA


DatePosted 29 Days Ago bot
NVIDIA Principal Software Engineer - Inference as a Service
Experience: Highly Experienced
Pattern: full-time
Country:
apply Apply Now
Salary:
Status:

Job

Copy Link Report
degreePhD
loacation US, CA, Santa Clara, United States Of America
loacation US, CA, Santa ..........United States Of America

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world. We are seeking a Principal Software Engineer to join our Software Infrastructure Team in Santa Clara, CA. This team is at the heart of the NVIDIA AI Factory initiative, building and maintaining the core infrastructure that powers our closed and open source AI models. In this role, you will be a key leader in designing and developing our Inference as a Service platform, creating the systems that manage GPU resources, ensure service stability, and deliver high-performance, low-latency inference at a massive scale. What you'll be doing: Lead the design and development of a scalable, robust, and reliable platform for serving AI models for inference as a service. Architect and implement systems for dynamic GPU resource management, autoscaling, and efficient scheduling of inference workloads. Build and maintain the core infrastructure, including load balancing and rate limiting, to ensure the stability and high availability of inference services. Optimize system performance and latency for various model types, from large language models (LLMs) to computer vision models, ensuring high-throughput and responsiveness. Develop tools and frameworks for real-time observability, performance profiling, and debugging of inference services. What we need to see: 15+ years of software engineering experience with deep expertise in distributed systems or large-scale backend infrastructure. BS, MS, or PhD in Computer Science, Electrical/Computer Engineering, Physics, Mathematics, other Engineering or related fields (or equivalent experience) Strong programming skills in Python, Go, or C++ with a track record of building production-grade, highly available systems. Proven experience with container orchestration technologies like Kubernetes. A deep understanding of system architecture for high-performance, low-latency API services. Experience in designing, implementing, and optimizing systems for GPU resource management. Familiarity with modern observability tools (e.g., DataDog, Prometheus, Grafana, OpenTelemetry). Demonstrated experience with deployment strategies and CI/CD pipelines. Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment. Ways to stand out from the crowd: Experience with specialized inference serving frameworks. Open-source contributions to projects in the AI/ML, distributed systems, or infrastructure space. Hands-on experience with performance optimization techniques for AI models, such as quantization or model compression. Expertise in building platforms that support a wide variety of AI model architectures. Strong understanding of the full lifecycle of an AI model, from training to deployment and serving. NVIDIA is widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and passionate people on the planet working for us. If you're creative and autonomous, we want to hear from you! Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 248,000 USD - 391,000 USD. You will also be eligible for equity and benefits . Applications for this job will be accepted at least until February 13, 2026. This posting is for an existing vacancy.  NVIDIA uses AI tools in its recruiting processes. NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Other Ai Matches

Senior Research Scientist, System Software and I/O Architecture Applicants are expected to have a solid experience in handling System Software and I/O Architecture related tasks
Data Center Engineer, HPC and AI Applicants are expected to have a solid experience in handling HPC and AI related tasks
Senior Inference Technical Product Marketing Manager - Accelerated Computing Applicants are expected to have a solid experience in handling Job related tasks
Physical Design Backend Engineer Applicants are expected to have a solid experience in handling Job related tasks
Senior Full-Stack Software Engineer - VLSI Tools Applicants are expected to have a solid experience in handling Job related tasks
Senior Software Architect, Humanoid Robotics Applicants are expected to have a solid experience in handling Humanoid Robotics related tasks
Senior System Software Engineer Platform - Server Embedded Firmware Applicants are expected to have a solid experience in handling Job related tasks
Senior Deep Learning Scientist, Multimodal Conversational AI Applicants are expected to have a solid experience in handling Multimodal Conversational AI related tasks
Senior Mask Designer Applicants are expected to have a solid experience in handling Job related tasks
Senior System Software Engineer - Power and Performance Applicants are expected to have a solid experience in handling Job related tasks
Senior Product Marketing Manager - Data Processing Applicants are expected to have a solid experience in handling Job related tasks
Firmware System Integration Engineer Applicants are expected to have a solid experience in handling Job related tasks
Senior Director, APAC Test Operations Applicants are expected to have a solid experience in handling APAC Test Operations related tasks
Senior Software Engineer, AI Platform Applicants are expected to have a solid experience in handling AI Platform related tasks
Senior Software Test Development Engineer - Deep Learning Applicants are expected to have a solid experience in handling Job related tasks
Senior Web Infrastructure Engineer Applicants are expected to have a solid experience in handling Job related tasks
Performance Engineering Intern, Platform Performance and Power - Summer 2026 Applicants are expected to have a solid experience in handling Platform Performance and Power - Summer 2026 related tasks
Software Engineering Intern, Neural Reconstruction - Summer 2026 Applicants are expected to have a solid experience in handling Neural Reconstruction - Summer 2026 related tasks
Firmware Engineer – Ethernet Switching Applicants are expected to have a solid experience in handling Job related tasks
Senior Architect Applicants are expected to have a solid experience in handling Job related tasks
remote-jobserver Remote
Senior Manager, Embedded Sales Applicants are expected to have a solid experience in handling Embedded Sales related tasks
Senior Engineering Build Manager - Autonomous Vehicles Applicants are expected to have a solid experience in handling Job related tasks
Senior Networking Software Architect Applicants are expected to have a solid experience in handling Job related tasks