Note: The job is a remote job and is open to candidates in USA. Netflix is rebuilding its AI/ML model-serving platform to support real-time, asynchronous, and agentic workloads across in-house infrastructure, third-party hosting, and external LLM providers. The Staff/Principal Software Engineer will set technical direction for this platform, improve inference efficiency and reliability, and support AI capabilities across member-facing products and internal tools.
Responsibilities
- You will tackle the efficiency problems that most directly drive cost per inference and capacity: disaggregated serving, intelligent routing across models and providers, speculative decoding, prompt caching, and multi-GPU inference
- You will lead the technology behind some of Netflix’s largest bets, embedding AI capabilities into member-facing products at global scale
- You will also build durable, resumable execution for long-running agentic workloads
- This is a rare very senior level opportunity to set and own the technical direction for a platform at the intersection of massive production scale, aggressive cost/ utilization engineering, and genuinely unsolved problems in GenAI serving by leading the technical architecture and directions for a growing engineering organization
Skills
- Have a BS/MS/PhD in Computer Science, Applied Math, Engineering, or a related field
- With relevant experience in the AI/ ML Infrastructure and distributed systems for 12+ years
- Influence without authority to guide architectural decisions and alignment across multi-teams through trust, data, and demonstrated expertise
- Shape long-term technical roadmaps (typically looking 2 to 3 years ahead) and ensure scalability, maintainability, and business alignment
- Work closely with product managers, operations, and executive stakeholders to balance technical trade-offs with business priorities
- Clear Communication skill to translate dense technical concepts for non-technical stakeholders and produce clear documentation, architecture decision records (ADRs), and proposals
- Have experience with high-traffic distributed services and infrastructure for AI/ ML model inference, with a focus on high availability, security, and performance
- Have experience building scalable model-serving solutions for Foundation Models that powers generative and predictive AI applications with practical skills in reducing latency and costs
- Familiar with deploying AI/ ML models using platform and solutions such as NVIDIA Triton Inference Server, vLLM, or SGLang, and scaling them via Kubernetes
- Have experience working with public cloud infrastructure providers such as AWS, Azure, or GCP
- Have open-source contributions to projects in the AI/ML, distributed systems, or infrastructure space
- Hands-on experience with building AI models, and have explored optimization techniques for AI models, such as quantization, model compression, or model pruning
- Proficient in leveraging AI coding tools in software development
- Have experience working with model gateway and intelligent routing for choosing the right AI models based on context of the requests
- Exposure to deploying multi-adapters model serving, disaggregated prefill/decode, or large-scale KV cache management
- Working with evaluation and benchmarking tools for LLMs and AI Agents
- Knowledge of GPU topology, NVIDIA CUDA programming, and machine learning compilers
Benefits
- Health Plans
- Mental Health support
- A 401(k) Retirement Plan with employer match
- Stock Option Program
- Disability Programs
- Health Savings and Flexible Spending Accounts
- Family-forming benefits
- Life and Serious Injury Benefits
- Paid leave of absence programs
- Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off
- Full-time salaried employees are immediately entitled to flexible time off
- Remote work in the United States
Company Overview
Company H1B Sponsorship