Note: The job is a remote job and is open to candidates in USA. Certa.ai is an AI-first third-party operating system that helps enterprises manage their vendors and partners. They are seeking an experienced AI Engineer to develop and deploy AI-driven solutions, focusing on generative AI and LLM technologies, to enhance their enterprise platform.
Responsibilities
- Design & ship AI features: Lead the design, development, and deployment of generative AI and LLM-powered services — intelligent chatbots, AI-driven recommendations, workflow automation - that deliver engaging, human-centric experiences
- Build RAG pipelines: Design, implement, and continuously optimize end-to-end RAG pipelines (data ingestion and parsing, chunking, vector indexing, prompt engineering) so our systems retrieve and use knowledge accurately
- Build LLM agents: Develop and refine LLM-based agentic systems for complex, multi-step tasks - incorporating planning, memory, and tool use, and applying emerging best practices to make agents more reliable
- Evaluate & iterate: Rigorously evaluate models and pipelines on accuracy, latency, and hallucination rate, using thorough testing and user feedback to improve prompts, parameters, and data processing
- Engineer for production: Write clean, maintainable, testable code with strong monitoring and logging, ensuring AI components are scalable and fit the overall system architecture
- Collaborate & innovate: Partner with product, design, and engineering to integrate AI seamlessly into products, mentor teammates on generative AI best practices, and continuously explore new AI advancements to bring into the platform
Skills
- 3+ years (mid-level) / 5+ years (senior) of backend engineering
- Expert Python
- Scalable API/service design
- Strong system design
- Experience deploying on AWS or similar cloud platforms
- Proven experience building real products on LLMs/generative AI (chatbots, semantic search, AI assistants)
- Hands-on RAG and agent work
- Solid grasp of model behavior and failure modes
- Strong in prompt design, function calling/structured outputs, tool use, context-window management
- Pragmatic model/provider trade-offs (latency, cost, context, safety)
- Product-aligned evaluation (golden sets, prompt unit tests, offline checks, online A/Bs)
- Fundamentals of embeddings, tokenization, vector search, and transformers
- Hands-on with LLM orchestration libraries (LangChain, LlamaIndex) and vector databases (Pinecone, Chroma, Milvus)
- Able to productionize LLM/RAG services as high-availability, low-latency backends on AWS (ECS/EKS/Lambda, API Gateway/ALB, S3, DynamoDB/Postgres, OpenSearch, SQS/SNS/Step Functions, KMS, VPC)
- Strong observability, security (PII handling, encryption, least-privilege IAM)
- Progressive delivery (blue/green, canary, feature flags)
- Cost/latency strategies like caching, batching, streaming, and multi-provider fallbacks
- Excellent communication to explain complex AI concepts to non-technical stakeholders
- Self-directed, 'figure it out' approach to troubleshooting across the stack and rapid prototyping in a fast-paced environment
- Enterprise B2B SaaS experience
- Full-stack/frontend depth (modern frameworks or Node.js)
- Complex multi-agent or multi-modal AI
- Early-stage startup experience
- Active involvement in the AI community (open source, research, or blogging)
Benefits
- 49 days paid time off
- PTO, full benefits, retirement and wellness plans.
- Professional development support
- Remote workspace setup bonus
- Annual company offsites in exciting locations around the world
Company Overview
Company H1B Sponsorship