Note: The job is a remote job and is open to candidates in USA. Cohere is the leading security-first enterprise AI company, building cutting-edge foundation AI models and products. As an Engineering Manager for GPU Infrastructure, you will lead a team of engineers to develop and operate the superclusters that support AI models, ensuring infrastructure reliability and collaborating with cross-functional teams.
Responsibilities
- Lead and mentor a team of engineers specializing in GPU infrastructure, fostering a culture of technical excellence and continuous improvement
- Manage performance, career development, and hiring for team members
- Conduct regular 1:1s and team meetings to ensure alignment and address challenges
- Provide technical guidance and support to team members on complex infrastructure problems
- Define and execute the technical roadmap for GPU cluster deployment, optimization, and scaling
- Oversee the implementation of topology-aware scheduling, hardware fault detection, and performance optimization systems
- Collaborate with cloud providers to validate and deploy new GPU architectures
- Ensure infrastructure reliability, scalability, and security across all GPU environments
- Partner with AI researchers to understand emerging infrastructure needs and translate them into robust solutions
- Work with the Foundations team on training software stack adaptation for new GPU architectures
- Coordinate with Capacity EPM on delivery timelines and resource planning
- Interface with Legal and Security teams on compliance requirements
- Collaborate with other infrastructure teams on shared goals and dependencies
- Establish observability and monitoring frameworks for GPU utilization, performance, and reliability
- Implement infrastructure-as-code practices and automation for cluster provisioning
- Drive cost optimization initiatives while maintaining performance standards
- Manage vendor relationships and contract negotiations for hardware and cloud services
- Ensure documentation is comprehensive, up-to-date, and accessible to stakeholders
Skills
- Experience managing engineering teams with a focus on technical mentorship and growth
- Strong communication skills to translate complex technical concepts for diverse audiences
- Ability to make data-informed decisions under pressure
- Experience working in remote, distributed teams
- Commitment to fostering an inclusive and collaborative team culture
- Deep expertise in ML/HPC infrastructure: GPU/TPU clusters, distributed training frameworks (JAX, PyTorch, TensorFlow), and high-performance computing environments
- Proven experience with Kubernetes at scale: deployment, management, and troubleshooting cloud-native clusters for AI workloads in multi-cloud environments
- Knowledge of infrastructure monitoring tools (Prometheus, Grafana)
- Familiarity with Terraform, ArgoCD, or other IaC tools
- Experience with cost optimization and capacity planning for GPU infrastructure
- Track record of collaborating with AI researchers or ML engineers to solve infrastructure challenges
- Strong problem-solving abilities with a data-driven approach
- Passion for enabling AI research through robust infrastructure
- Collaborative mindset with a focus on cross-team success
- Willingness to learn and adapt in a fast-paced, evolving environment
Benefits
- A weekly lunch stipend of $75/£75 or equivalent in your local currency for lunch.
- Full health and dental benefits, including a separate budget for mental health.
- RRSP matching, 401K, Pension Scheme.
- 100% Parental Leave top-up for up to 6 months, for either parent.
- Annual enrichment benefits: Arts & culture, fitness/wellness, quality time, and a workspace improvement credit. Education & learning stipend for conferences, courses, and coaching.
- 6 weeks of paid vacation (30 working days!)
- Budget for traveling to other offices if you are remote, plus an annual company offsite.
- Cohere is remote-friendly. We have offices in Toronto, San Francisco, New York City, London, Paris, Montreal, and more coming soon.
- For those in the office: a daily lunch program, plenty of snacks, and regular community and social events.
- For those not near an office: a co-working benefit so you can work alongside others in your city.
- Everyone receives a $500 home office stipend to set up your workspace properly.
Company Overview
Company H1B Sponsorship