Note: The job is a remote job and is open to candidates in USA. EPAM Systems is building an AI Reliability Engineering capability within its Operational Intelligence practice. The Senior AI Reliability Engineer will instrument, monitor, evaluate, and harden production AI, ML, and LLM systems while defining AI-native reliability objectives and developing reusable practice accelerators, reference architectures, and client enablement materials.
Responsibilities
- Instrument production LLM, RAG and agentic applications with AI telemetry based on OpenTelemetry and APM-native AI monitoring
- Implement distributed tracing across multi-model chains, agent workflows and retrieval-augmented generation pipelines to profile systemic latency and failure points
- Define and measure AI-native SLIs and SLOs: time to first token (TTFT), throughput, error and refusal rates, cost per request, semantic drift, hallucination boundaries, contextual accuracy
- Set up structured semantic logging and prompt/response monitoring for quality analysis
- Build and run evaluation loops for output quality and safety — golden sets, LLM-as-a-judge, Ragas/DeepEval-style frameworks — and wire them into CI/CD and runtime
- Track token-based cloud spend, model API rate limits and quota consumption; drive AI cost optimization
- Configure AI gateways for API load balancing, failover and fallback models across multiple LLM providers
- Implement guardrails: prompt injection and jailbreak filtering, output compliance, bias and safety constraints
- Design detection, triage, restore and problem management workflows for AI incidents; integrate autonomous AI agents into RCA to parse logs, form hypotheses and correlate state changes
- Support rollback, canary and fail-safe patterns for model, prompt and configuration releases; maintain reproducibility through versioning of data, code, prompts and models
- Build practice accelerators, reference architectures and internal enablement materials; support presales and client assessments
Skills
- 4+ years in SRE, DevOps, platform or observability engineering, including hands-on work with production AI/ML or LLM workloads
- Solid SRE fundamentals: Golden Signals, SLI/SLO definition, error budgets and burn rate, incident lifecycle, ITIL basics
- Strong Python for instrumentation, automation and evaluation tooling
- Practical experience with OpenTelemetry and at least one APM/observability platform: New Relic, Datadog, Grafana LGTM stack, Splunk or Elastic
- Hands-on production experience with at least one cloud platform (Azure preferred, AWS or GCP) and Kubernetes
- Working understanding of LLM application architecture: prompts, embeddings and vector stores, RAG, agent orchestration (LangChain / LangGraph or equivalent)
- MLOps awareness: model lifecycle (training vs. inference), model endpoints, containerization, deployment and rollback patterns
- Infrastructure as Code with Terraform; CI/CD experience (Azure DevOps, GitLab CI, GitHub Actions)
- B2+ English — the role is client-facing and requires clear written and spoken technical communication
- Experience with AI-specific observability and evaluation tooling: Traceloop/OpenLLMetry, Langfuse, Arize Phoenix, Ragas, DeepEval, MLflow
- Distributed inference serving at scale: vLLM, KServe, Ray Serve, Kubernetes-native LLM orchestration, GPU capacity planning
- AI security: OWASP LLM Top 10, prompt injection defense, guardrail frameworks (NeMo Guardrails, Llama Guard)
- Databricks (incl. Mosaic AI / MLflow) or Azure AI Foundry experience
- Certifications: Anthropic Claude, Azure AI Engineer, AWS ML Specialty, Databricks GenAI
- FinOps for AI workloads — token and GPU cost modeling
- Data Reliability Engineering background (data quality, pipeline SLOs) — AI reliability starts with data reliability
- Mentoring or team lead experience
Benefits
- A genuinely new discipline inside EPAM — you define how it is done, not follow an existing runbook
- Engineering focus without 24/7 on-call rotation
- Funded certification and enablement tracks (Anthropic/Claude, Databricks, AI & Data Observability learning paths)
- Cross-client exposure and a direct path into presales and solution engineering
- Work on a flexible schedule remotely or from any of our comfortable offices or coworking spaces in Ukraine
- Receive the necessary equipment to perform your work tasks
- Change projects and technology stacks within EPAM
- Gain experience in various business domains (Insurance, E-commerce, Healthcare, Finance, Travelling, Media, Artificial Intelligence, and more)
- Relocation opportunities may be available for eligible candidates, depending on the role and openings at other EPAM locations
- Participate in volunteer, charity programs and communities (both technical and interest-based)
- Plan your individual career path together with your manager
- Receive regular feedback from colleagues
- Improve your English for free with certified teachers (Speaking Clubs, client interview preparation courses, etc.)
- Get the opportunity to undergo free training and certification in AWS, GCP, or Azure Clouds
- Use the internal E-learn training program (18,200+ specialized training and mentoring programs)
- Access corporate accounts on LinkedIn Learning, Get Abstract and other partner resources
- Study at EPAM Solution Architecture School with the instructors who are practicing architects
- Develop as a leader, join Delivery Management, Resource Management, Leadership Essentials school and more
- Participate in internal communities (500+ meetups, technical discussions, brainstorming sessions, online events and conferences annually)
- Vacation and sick leave (including a sick leave without a medical certificate)
- A wide range of Voluntary Medical Insurance programs providing both medical treatment and various preventive options (including sports activities)
- Medical insurance for family members at corporate rates
- Company support during significant life events (childbirth or adoption, marriage, etc.)
- Support for psychological comfort: discounts on services from mental health specialists or coaches, thematic training
- E-kids program - a free programming language training program for EPAMers' children
Company Overview
Company H1B Sponsorship