Note: The job is a remote job and is open to candidates in USA. AddSource is seeking a Senior AI Engineering Manager specialized in Machine Learning. The role involves designing, building, and deploying AI-powered features while ensuring reliable and scalable solutions throughout the product lifecycle.
Responsibilities
- End-to-End AI Feature Ownership
- Design and implement AI-powered features (LLM workflows, copilots, and agent-based systems with tool use and multi-step reasoning)
- Own the Full Lifecycle: prototyping evaluation production deployment iteration
- Ensure solutions are reliable, performant, and aligned with product needs
- AI System Implementation
- Build and optimize prompt pipelines for specific use cases
- Build retrieval systems (embeddings, chunking, ranking)
- Implement RAG-based workflows where needed
- Iterate on outputs to improve quality, accuracy, and consistency
- Design scalable and cost-efficient AI architectures for production workloads
- Select and evaluate models (hosted vs open-source) based on use case constraints
- Agent-Based Systems (AgentCore)
- Design and build agentic workflows capable of multi-step reasoning and decision-making
- Integrate agents with tools, APIs, and internal systems to perform real-world actions
- Implement planning, execution, and reflection loops for complex tasks
- Manage context, memory, and state across multi-step interactions
- Balance deterministic workflows vs. agent autonomy for reliability and control
- Experimentation & Evaluation
- Run structured experiments to compare approaches (prompting, retrieval, models)
- Define and track key metrics for AI performance (quality, latency, cost)
- Debug and improve non-deterministic system behavior
- Build and maintain evaluation datasets and benchmarks
- Implement automated evaluation pipelines for continuous improvement
- Collaboration & Contribution
- Drive technical direction and influence AI adoption across teams
- Partner with product managers and designers to scope AI features
- Contribute to shared patterns and reusable components
- Participate in code reviews and design discussions
- Support and mentor mid-level engineers where needed
- AI Reliability, Safety & Governance
- Design guardrails to ensure safe and reliable AI behavior
- Mitigate hallucinations, prompt injection, and model misuse
- Ensure compliance with data privacy and enterprise requirements
- Implement monitoring and observability for AI systems in production
- Implement guardrails for agent actions (tool access control, execution boundaries)
- Prevent failure cascades in multi-step agent
Skills
- Minimum 10+ Years
- Engineering Degree BE/ME/BTech/MTech/BSc/MSc
- Strong understanding of LLM capabilities and limitations
- Experience with prompt engineering and structured output design
- Hands-on experience with embeddings and vector search
- Familiarity with RAG architectures and when to apply them
- Experience designing agent-based architectures (AgentCore concepts)
- Understanding of tool use, planning strategies, and memory mechanisms in LLM systems
- 5+ years of related work experience
- Solid backend/system design fundamentals
- Experience building and deploying production-grade systems
- Ability to debug complex issues, including probabilistic outputs
- Comfort working with APIs, pipelines, and data flows
- Ability to translate user needs into effective AI solutions
- Strong intuition for balancing quality, latency, and cost
- Focus on delivering measurable product impact
- Communicates clearly across engineering and product teams
- Contributes to team knowledge and shared practices
- Technical certification in multiple technologies is desirable
- Evaluate agent performance across multi-step tasks (task success rate, error propagation)
- Debug and optimize agent decision-making and tool selection behavior
Company Overview