Note: The job is a remote job and is open to candidates in USA. Yahoo is a global technology company connecting its brands and partners with hundreds of millions of people through large-scale digital platforms. Yahoo is seeking a Principal Engineer to lead the architecture and development of AI-powered data intelligence platforms, including distributed data systems, retrieval-augmented generation, and agentic AI capabilities. The role will establish technical direction, build reusable platform capabilities, and advance reliable AI-assisted software development across the organization.
Responsibilities
- Define technical vision and architecture for Yahoo's AI-powered data intelligence platform
- Architect petabyte-scale batch and streaming systems with Beam, Dataflow, Spark, and Kafka
- Build production RAG and multi-agent systems: semantic metadata integration, agent orchestration
- Establish evaluation frameworks — retrieval quality, agent accuracy, hallucination rate, operational safety — with automated benchmarking, so we ship AI we can trust
- Use AI development tools to accelerate design, implement the bar for how the org validates AI-generated code and agent plans
- Drive adoption of RAG and agentic patterns across engineers
- Partner with product, governance, infrastructure, and exit-form priorities
Skills
- 10+ years building distributed systems and large-scale data platforms, or equivalent experience
- Degree in CS, AI/ML, Engineering, Math, or related field
- Deep Python expertise and strong software engineering fun
- Production experience with batch and streaming processing (Kafka, or equivalent)
- Shipped and operated RAG applications in production; search, and reranking
- Track record evaluating and validating AI-generated code correctness and operational safety
- Strong communication and technical leadership across cross-functional stakeholders
- Agentic AI frameworks (LangGraph, CrewAI, AutoGen, Semant)
- GCP: Vertex AI, Gemini, BigQuery, Vector Search, Dataflow
- Enterprise metadata, lineage, governance, or observability
- Knowledge graphs and semantic reasoning systems
- Daily use of AI pair-programming tools (Claude, Copilot, g workflows)
- Led large technical initiatives influencing strategy across multiple teams
Benefits
- Flexible hybrid work options
- Incentive compensation opportunities in the form of discretionary annual bonus or commissions
- Healthcare
- 401k
- Backup childcare
- Education stipends
- Employee resource groups (ERGs) with programs, events and fellowship that help educate, support and create a workplace where all feel welcome
Company Overview
Company H1B Sponsorship