Note: The job is a remote job and is open to candidates in USA. Upscale AI is building high-performance infrastructure for demanding artificial intelligence workloads. The Technical Support Senior Staff Engineer will own network bring-up and performance for GPU AI clusters, tune switching ASIC pipelines, and ensure fabric performance during customer proof-of-concepts. The role also involves debugging software and infrastructure, developing automation and observability tools, validating networking platforms, and collaborating with engineering teams and customers.
Responsibilities
- Design, develop, and maintain features and enhancements for the SONiC NOS platform
- Debug, troubleshoot, and resolve issues on SONiC platforms
- Develop and execute debugging and troubleshoot infrastructure
- Collaborate closely with cross-functional teams including hardware engineers and Test teams
- Participate in code reviews, architecture discussions, and documentation efforts
- Develop support strategies to root-cause Networking ASICs and Networking Systems issues
- Develop debugging tools, for Traffic monitoring and Performance measurements
- Debug issues across software, Linux systems, networking stacks, and distributed infrastructure
- Be a point of contact for customer deployments, integration testing, proof-of-concepts, and field issue resolution
- Build tools that improve deployment efficiency, observability, telemetry, and automated testing
- Collaborate with software, infrastructure, QA, and product teams to identify root causes and deliver robust solutions
- Contribute to backend services, APIs, orchestration components, and infrastructure automation
- Document technical findings and communicate effectively with engineering teams and customers
Skills
- · Bachelor's or master's degree in computer science, Electrical Engineering, or a related field
- · Minimum of 10 years of work experience is required, with at least 5 years of hands-on SONiC or equivalent Network Operating
- · Strong programming skills in Python, Go, or a similar language
- · Solid understanding of Linux, TCP/IP networking, routing, switching, VLANs, and network troubleshooting
- · Experience with PTF (Packet Test Framework) and SPyTest for network validation
- · Familiarity with Linux internals, docker containers
- · Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment
- · Knowledge of network ASICs and switch hardware architecture is mandatory
- · Excellent written and verbal communication skills
- · Ability to thrive in a collaborative, fast-paced startup environment with a strong sense of ownership
- · Data-center networking with hands-on switch ASIC tuning and platform bring-up
- · Deep buffer/queueing knowledge (headroom, xon/xoff, dynamic thresholds, VOQ vs shared pools)
- · SONiC, Enterprise-OS
- · Optics & PHY: PAM4 signal integrity, FEC modes, DOM/RS-FEC counters, AN/LT quirks, DAC/AOC selection
- · Tooling: ethtool, devlink, mlnx\_qos, perfquery, switch telemetry (INT/sFlow/ERSPAN), gNMI/REST, Prometheus
- · Benchmarking: perftest, nccl-tests, iPerf3, traffic generators; reading queue stats, ECN marks, and CNP behavior
- · EVPN/VXLAN leaf-spine for AI pods; flowlet or latency-aware hashing
- · BlueField DPU offloads, GPUDirect RDMA; NIC QoS (DSCP-to-TC, PFCx, GEARBOX/FW)
- · Storage fabrics for AI (NFS-RDMA, NVMe-oF) and their QoS interactions
- · Python/Ansible for templated ASIC profiles; Git workflows for config promotion
- At least 5 years of hands-on SONiC or equivalent Network Operatinga0 a0 a0System (NOS) development experience preferred
Benefits
- Learning from experienced engineers in a fast-paced startup environment
Company Overview
Company H1B Sponsorship