Note: The job is a remote job and is open to candidates in USA. Cobalt builds expert data and evaluation infrastructure for AI developers. The company is seeking a Principal Distributed Systems Engineer to design, implement, test, and refine realistic command-line distributed systems tasks that evaluate frontier AI models’ systems engineering capabilities.
Responsibilities
- Design self-contained command line tasks drawn from real distributed systems work, such as finding a race condition or deadlock in a concurrent service, fixing a consistency bug in a replicated data store, diagnosing a failure from logs and traces across several services, repairing a broken consensus or leader election implementation, tuning a service that degrades under load, and recovering a cluster from a corrupted or partially failed state
- Build the task environment, typically several cooperating processes or containers on a single machine, write a reference solution, and write automated tests that verify whether a solution is correct. Because concurrency and timing bugs are hard to reproduce, tests need to trigger the failure reliably rather than intermittently
- Test your task against a frontier AI model and refine it until the model fails for substantive reasons rather than because of ambiguity, trick wording, or flaky tests
- Respond to reviewer feedback until the task is accepted
Skills
- A PhD in computer science, distributed systems, or a closely related field
- Academic or industry experience building or operating distributed systems, such as databases, storage systems, messaging systems, schedulers, or large-scale service infrastructure
- At least one publication, either academic (for example a peer-reviewed paper at a recognized systems venue) or professional (for example a conference talk, a detailed engineering article, or a widely used open-source system)
- Strong programming skills in at least one systems language, such as Go, Rust, C++, or Java
- Fluency in the Linux command line, shell scripting, Docker, networking tools, and Git
- The ability to write precise task specifications that another engineer could follow without asking questions
Benefits
- Expand your influence through evaluation projects, advisory roles, and research collaborations, while deepening your understanding of how frontier models are trained and assessed.
- Collaborate with researchers from leading institutions and labs on high-impact, flexible work.
- Flexible 10 to 40 hour weeks that fit around your research position and your life.
Company Overview