Note: The job is a remote job and is open to candidates in USA. Upscale AI builds high-performance infrastructure for next-generation artificial intelligence workloads. The Technical Support Principal Engineer – AI Network will lead network bring-up and performance for GPU AI clusters, develop and debug SONiC-based networking platforms, and resolve complex software, hardware, and infrastructure issues with engineering teams and customers.
Responsibilities
- Design, develop, and maintain features and enhancements for the SONiC NOS platform
- · Debug, troubleshoot, and resolve issues on SONiC platforms
- · Develop and execute debugging and troubleshoot infrastructure
- · Collaborate closely with cross-functional teams including hardware engineers and Test teams
- · Participate in code reviews, architecture discussions, and documentation efforts
- · Develop support strategies to root-cause Networking ASICs and Networking Systems issues
- · Develop debugging tools, for Traffic monitoring and Performance measurements
- · Debug issues across software, Linux systems, networking stacks, and distributed infrastructure
- · Be a point of contact for customer deployments, integration testing, proof-of-concepts, and field issue resolution
- · Build tools that improve deployment efficiency, observability, telemetry, and automated testing
- · Collaborate with software, infrastructure, QA, and product teams to identify root causes and deliver robust solutions
- · Contribute to backend services, APIs, orchestration components, and infrastructure automation
- · Document technical findings and communicate effectively with engineering teams and customers
Skills
- · Bachelor's or master's degree in computer science, Electrical Engineering, or a related field
- · Minimum of 12 years of work experience is required, with at least 3 years of hands-on SONiC or equivalent Network Operating System (NOS) development experience preferred
- · Strong programming skills in Python, Go, or a similar language
- · Solid understanding of Linux, TCP/IP networking, routing, switching, VLANs, and network troubleshooting
- · Experience with PTF (Packet Test Framework) and SPyTest for network validation
- · Familiarity with Linux internals, docker containers
- · Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment
- · Knowledge of network ASICs and switch hardware architecture is mandatory
- · Excellent written and verbal communication skills
- · Ability to thrive in a collaborative, fast-paced startup environment with a strong sense of ownership
- · at least 3 years of hands-on SONiC or equivalent Network Operating System (NOS) development experience preferred
- · Data-center networking with hands-on switch ASIC tuning and platform bring-up
- · Proven RoCEv2 deployments at 200/400/800 G: ECN/PFC design, DCQCN tuning, DSCP/PCP mapping, queue/WRR shaping
- · Deep buffer/queueing knowledge (headroom, xon/xoff, dynamic thresholds, VOQ vs shared pools)
- · SONiC (buffers.json, qos.json, pfcwd, ecn), Enterprise-OS (class-map/type qos & network-qos, policy-map, queuing)
- · Optics & PHY: PAM4 signal integrity, FEC modes, DOM/RS-FEC counters, AN/LT quirks, DAC/AOC selection
- · Tooling: ethtool, devlink, mlnx\_qos, perfquery, switch telemetry (INT/sFlow/ERSPAN), gNMI/REST, Prometheus
- · Benchmarking: perftest, nccl-tests, iPerf3, traffic generators; reading queue stats, ECN marks, and CNP behavior
- · EVPN/VXLAN leaf-spine for AI pods; flowlet or latency-aware hashing
- · BlueField DPU offloads, GPUDirect RDMA; NIC QoS (DSCP-to-TC, PFCx, GEARBOX/FW)
- · Storage fabrics for AI (NFS-RDMA, NVMe-oF) and their QoS interactions
- · Python/Ansible for templated ASIC profiles; Git workflows for config promotion
Company Overview
Company H1B Sponsorship