Note: The job is a remote job and is open to candidates in USA. EPAM Systems is building a self-serve Enterprise AI Gateway that standardizes and governs how teams use large language models. The Lead AI OPS Engineer will design automation-first onboarding, access controls, provisioning, model deployment, platform observability, and production reliability while supporting teams that use the platform.
Responsibilities
- Design self-service onboarding for teams and agents, from initial request through to functional access
- Automate key provisioning, rotation, and permission updates
- Tune rate limits and quotas as adoption and usage increase
- Deploy and configure models including Vertex AI endpoints, vendor fallbacks, and Model Armor
- Operate MCP servers behind the gateway under the same governance rules as models
- Develop Python extensions that integrate the gateway with other enterprise systems
- Maintain the full platform as code using Terraform, Helm, and Jenkins
- Monitor and protect production health across availability, latency, and cost with alerting
- Assist teams using the platform and convert recurring questions into automation
Skills
- Proven 5+ years of experience using Python for automation, extensions, and integrations
- Solid 5+ years of SRE experience with a track record of keeping production systems reliable
- Deep expertise in Kubernetes, preferably on GKE
- Hands-on experience with Google Vertex AI, especially endpoints for model serving and Model Armor
- Practical proficiency with Terraform, Helm, and CI/CD using Jenkins
- Working knowledge of GenAI/Agentic AI concepts (patterns, frameworks, protocols)
- Very strong communication skills with the ability to explain platform behavior clearly to teams who use it daily
- English proficiency at B2 level (Upper-Intermediate) or higher
- Production exposure to an AI gateway (LiteLLM, EPAM DIAL or similar) is highly appreciated
- Familiarity with GCP beyond GKE and Vertex AI, such as BigQuery, Cloud Run, and IAM
- Experience building AI agents, for example using Google's Agent Development Kit (ADK)
- Experience with AWS Bedrock
Benefits
- International projects with top brands
- Work with global teams of highly skilled, diverse peers
- Healthcare benefits
- Employee financial programs
- Paid time off and sick leave
- Upskilling, reskilling and certification courses
- Unlimited access to the LinkedIn Learning library and 22,000+ courses
- Global career opportunities
- Volunteer and community involvement opportunities
- EPAM Employee Groups
- Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn
Company Overview
Company H1B Sponsorship