Note: The job is a remote job and is open to candidates in USA. NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing the state of AI model deployment. The role involves shaping software for sophisticated AI systems and optimizing frameworks for scalable deployment on NVIDIA GPUs.
Responsibilities
- Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software
- Drive the strategy, roadmap, and execution of NVIDIA’s inference frameworks engineering, focusing on Client AI
- Partner with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators
- Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications
- Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications (NIXL, NCCL, NVSHMEM)
- Represent the team in roadmap and planning discussions, ensuring alignment with NVIDIA’s broader AI and software strategies
- Foster a culture of technical excellence, open collaboration, and continuous innovation
Skills
- MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a related field
- 6+ overall years of software development experience, including 3+ years in technical leadership or engineering management
- Strong background in C/C++ software design and development; proficiency in Python is a plus
- Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization
- Proven record of deploying or optimizing deep learning models in production environments
- Experience leading teams using Agile or collaborative software development practices
- Significant open-source contributions to deep learning or inference frameworks such as PyTorch, vLLM / SGLang, Triton, or TensorRT-LLM
- Deep understanding of multi-GPU communications (NIXL, NCCL, NVSHMEM) and distributed inference architectures
- Expertise in performance modeling, profiling, and system-level optimization across CPU and GPU platforms
- Proven ability to mentor engineers, guide architectural decisions, and deliver complex projects with measurable impact
- Publications, patents, or talks on LLM serving, model optimization, or GPU performance engineering
Benefits
- Equity
- Comprehensive benefits package
- Eligible for equity and benefits
Company Overview
Company H1B Sponsorship