Note: The job is a remote job and is open to candidates in USA. Acuity Analytics is a business with a strong global reputation, and they are seeking a Senior AI & Data Engineer (GenAI & LLM Evaluation) to help build and improve AI-powered data products. The role involves developing scalable data pipelines and deploying production-ready AI solutions while working on cutting-edge Generative AI technologies.
Responsibilities
- Build and maintain scalable data pipelines and workflows
- Develop evaluation frameworks for GenAI and LLM applications
- Create and maintain golden datasets for model testing
- Evaluate and benchmark LLM performance
- Build ML-assisted tools to improve delivery and automation
- Deploy production-ready AI solutions
- Work closely with delivery teams to solve technical challenges
- Continuously improve data quality, automation, and operational efficiency
Skills
- 5+ years' experience in Data Engineering, Machine Learning, AI Engineering, or Data Science
- Strong Python and SQL skills
- Hands-on experience with Generative AI and Large Language Models (LLMs)
- Experience evaluating LLMs using golden datasets, model benchmarking, or LLM-as-a-Judge techniques
- Experience building and deploying production-ready ML solutions
- Strong understanding of data pipelines, validation, and quality frameworks
- Experience integrating APIs and cloud-based AI services
- Strong problem-solving and analytical skills
- Comfortable owning solutions end-to-end
- Excellent communication and stakeholder management
- Experience working in Agile, cross-functional teams
- Consulting or customer-facing experience is a plus
Company Overview