Note: The job is a remote job and is open to candidates in USA. GitLab is hiring a Principal Engineer to set technical direction and build the Duo Agent Platform, an orchestration layer for AI agents and workflows across the software lifecycle. The role spans architecture and hands-on implementation, platform reliability and governance, evaluation, cross-team alignment, mentoring, and hiring.
Responsibilities
- Define the technical direction for a significant area of Duo Agent Platform across several teams, and partner with Engineering Managers and Product to turn it into a roadmap with a six-month horizon
- Build the orchestration layer that makes agents and flows reliable: session lifecycle, tool invocation, context assembly, concurrency, retries, timeouts, and failure modes that degrade gracefully instead of stalling a developer's workflow
- Hands on work on driving topics throughout the whole organisation from POC level, architecture to full implementation
- Participate in Incident Management on-call rotation, to help meet availability goals for GitLab.com
- Strong product builder and ownership mentality. Understanding of product strategy, technical strategy, delivering customer value and iteration towards a successful product
- Write clear, opinionated design documents, architecture narratives, and decision records that let other teams make aligned decisions. Ensure the architecture holds across multi-tenant GitLab.com, Dedicated, and Self-Managed deployments, including self-hosted model configurations, and respects data governance boundaries in each
- Use the platform on our own work, find where it falls short of what we claim, and feed that back through improvements into the product
- Mentor Staff and Senior Engineers, and raise the bar for experimentation rigor and cross-team collaboration
- Work across the whole platform on different projects, features and teams. This reaches across:
- Design the extensibility surface so customers can build custom agents and flows, connect their own tools and internal systems, and bring external agents such as Claude Code and Codex into GitLab workflows without weakening the platform's guarantees
- Drive down the cost and latency of agentic requests, and work with Product on how model selection, routing, and credit consumption behave as usage grows
- Define the evaluation practice for the platform: offline benchmarks and online experiments that measure task completion, correctness, tool-call accuracy, latency, and cost per task, and that catch regressions before customers do
- Build the governance and safety surface with Security and Compliance: policy-driven controls over which agents and models may run where, permission and identity semantics that align to existing roles and group structure, human-in-the-loop checkpoints, traceability of every flow session, and audit that stands up to enterprise review
- Complete Interview Training and participate in hiring for the department as a technical interviewer or panel member
Skills
- 8+ years of software engineering experience, including 3+ years at Staff, Principal level or equivalent senior technical leadership
- Hands-on production experience building on large language models: agent loops, tool calling, context management, retrieval, prompt and model evaluation. Experience shipping this to real users, not only prototyping it
- Strong background in distributed systems, including service decomposition, fault tolerance, observability, and operational resilience under multi-tenant load
- A track record of defining evaluation frameworks and using benchmark and experiment results to make architecture decisions, rather than shipping on intuition
- Experience designing human-in-the-loop controls, safety guardrails, and responsible AI practices for systems that act on a customer's behalf
- Ability to drive alignment across Engineering, Product, Infrastructure, and Security on problems that start out ambiguous
- Experience mentoring senior engineers and shaping technical direction across multiple teams without direct authority
- Comfort working in one or more of Ruby, Go, Python, or TypeScript, and willingness to work across the stack the platform spans
- Excellent written and asynchronous communication, and the ability to work effectively in a fully remote, globally distributed organization
Benefits
- Benefits to support your health, finances, and well-being
- Flexible Paid Time Off
- Team Member Resource Groups
- Equity Compensation & Employee Stock Purchase Plan
- Growth and Development Fund
- Parental Leave
- All roles are remote
Company Overview