Note: The job is a remote job and is open to candidates in USA. Cohere is a security-first enterprise AI company building foundation AI models and end-to-end products for real-world business problems. The Site Reliability Engineer will develop, deploy, and operate the AI platform serving large language models through API endpoints. The role focuses on scalable, reliable, low-latency infrastructure, customized deployments, observability, resilience, and collaboration with internal teams and customers.
Responsibilities
- Build self-service systems that automate managing, deploying and operating services
- This includes our custom Kubernetes operators that support language model deployments
- Automate environment observability and resilience. Enable all developers to troubleshoot and resolve problems
- Take steps required to ensure we hit defined SLOs, including participation in an on-call rotation
- Build strong relationships with internal developers and influence the Infrastructure team’s roadmap based on their feedback
- Develop our team through knowledge sharing and an active review process
Skills
- 5+ years of engineering experience running production infrastructure at a large scale
- Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
- Experience with Kubernetes dev and production coding and support
- Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
- Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
- Experience in compute/storage/network resource and cost management
- Excellent collaboration and troubleshooting skills to build mission-critical systems, and ensure smooth operations and efficient teamwork
- The grit and adaptability to solve complex technical challenges that evolve day to day
- Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators), especially how they influence latency and throughput of inference
- Strong understanding or working experience with distributed systems
- Experience in Golang, C++ or other languages designed for high-performance scalable servers)
Benefits
- A weekly lunch stipend of $75/£75 or equivalent in your local currency for lunch.
- Full health and dental benefits, including a separate budget for mental health.
- RRSP matching, 401K, Pension Scheme.
- 100% Parental Leave top-up for up to 6 months, for either parent.
- Annual enrichment benefits: arts & culture, fitness/wellness, quality time, and a workspace improvement credit.
- Education & learning stipend for conferences, courses, and coaching.
- 6 weeks of paid vacation (30 working days!).
- Budget for traveling to other offices if you are remote, plus an annual company offsite.
- Cohere is remote-friendly.
- For those in the office: a daily lunch program, plenty of snacks, and regular community and social events.
- For those not near an office: a co-working benefit so you can work alongside others in your city.
- Everyone receives a $500 home office stipend to set up your workspace properly.
Company Overview
Company H1B Sponsorship