Note: The job is a remote job and is open to candidates in USA. Atlassian is a software company focused on collaboration products and AI innovation across its platforms. The Senior Machine Learning Systems Engineer will design and optimize large-scale model serving systems, including distributed infrastructure, inference engines, and low-level GPU optimizations. The role also involves building reliable deployment infrastructure and partnering with senior ML engineers to fine-tune and deploy open-source LLMs.
Responsibilities
- Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache)
- Optimize latency and throughput of model inference under real production workloads
- Build reliable, high-concurrency serving systems that serve billions of requests reliably
- Benchmark, fine-tune, and accelerate inference engines
- Create robust CI/CD infrastructure for seamless model deployment and inference engine updates
- Partner with senior ML engineers to fine‑tune open-source LLMs and deploy
Skills
- 5+ years of software engineering experience, 2+ years of system performance optimization experience
- Deep low-level systems programming (C/C++ or Rust)
- Experience with large-scale, high-concurrent production serving
- Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.)
- Strong background in system optimizations: batching, caching, load balancing, parallelism
- Low-level inference optimizations: GPU kernels
- Algorithmic inference optimizations: quantization, speculative decoding, distillation
- Experience with testing, benchmarking, and reliability of inference services
- Experience designing and implementing CI/CD infrastructure for inference
Benefits
- Atlassians can choose where they work – whether in an office, from home, or a combination of the two.
- Health and wellbeing resources
- Paid volunteer days
- This role may also be eligible for benefits, bonuses, commissions, and equity.
Company Overview
Company H1B Sponsorship