Note: The job is a remote job and is open to candidates in USA. EPAM Systems is seeking a Senior AI OPS Engineer to build and operate an enterprise AI Gateway that governs access to large language models. The role involves automating onboarding, permissions, provisioning, model deployment, MCP server management, infrastructure as code, and production reliability while supporting platform users.
Responsibilities
- Build self-service onboarding for teams and agents, from request to working access
- Automate key provisioning, rotation and permission changes
- Adapt rate limits and quotas as usage grows
- Roll out and configure models: Vertex AI endpoints, vendor fallbacks, Model Armor
- Run MCP servers behind the gateway, under the same rules as models
- Write Python extensions that connect the gateway to other enterprise systems
- Manage the whole platform as code with Terraform, Helm and Jenkins
- Keep production healthy: availability, latency and cost, watched by alerts
- Support the teams using it, and turn every repeat question into automation
Skills
- 3+ years of experience in Python for automation, extensions and integrations
- 3+ years of experience in SRE, with real experience keeping production systems healthy
- Deep expertise in Kubernetes, preferably on GKE
- Hands-on experience with Google Vertex AI, especially endpoints for model serving and Model Armor
- Proficiency in Terraform, Helm and CI/CD with Jenkins
- Working knowledge of GenAI/Agentic AI concepts (patterns, frameworks, protocols)
- Very good communication skills, with the ability to explain things clearly to teams using the platform every day
- English proficiency at B2 level or higher
- Production experience with an AI gateway (LiteLLM, EPAM DIAL or similar) is highly appreciated
- Familiarity with GCP beyond GKE and Vertex AI (BigQuery, Cloud Run, IAM)
- Experience building AI agents, for example with Google's Agent Development Kit (ADK)
- Experience with AWS Bedrock
Benefits
- International projects with top brands
- Work with global teams of highly skilled, diverse peers
- Healthcare benefits
- Employee financial programs
- Paid time off and sick leave
- Upskilling, reskilling and certification courses
- Unlimited access to the LinkedIn Learning library and 22,000+ courses
- Global career opportunities
- Volunteer and community involvement opportunities
- EPAM Employee Groups
- Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn
Company Overview
Company H1B Sponsorship