Note: The job is a remote job and is open to candidates in USA. EY is a globally connected professional services organization focused on building a better working world through assurance, consulting, tax, strategy, transactions, data, AI, and advanced technology. The AI Systems Engineer will own the delivery, model-serving, routing, governance, cost management, and observability layers of EY’s AI-native platform across cloud, on-premises, edge, and air-gapped environments. The role is responsible for operating AI delivery pipelines, inference systems, telemetry, and FinOps controls so workloads are secure, traceable, economical, and reliably deployed.
Responsibilities
- Supports DevOps and delivery for AI workloads: build and operate the CI/CD/CV pipelines that ship AI services, agents, and runtime components, including automated build, test, continuous verification, release, and rollback, so AI workloads are delivered repeatably and safely into every environment
- Own governance and discovery for AI assets, including service catalog/registry (Artifactory/Nexus, Harbor), experiment tracking and model metadata (MLflow), upstream registries/mirrors (HuggingFace/NGC), CVE/SBOM scanning (Trivy), lineage contracts (OpenLineage), and license management
- Own resource and cost management, including quotas and rate limits, cost attribution and utilization (Apptio/OpenCost/Kubecost), so AI execution stays economically bounded and controllable per tenant and engagement
- Own the full observability stack, including metrics (Prometheus/Mimir), logs (Loki), traces (Tempo/Jaeger), dashboards (Grafana), LLM debugging and evaluation (LangSmith/Langfuse), and SLA/alert notifications
- Own the OpenTelemetry collection layer, including multi-tenant receiver, exporters and queues (Kafka sink), DCGM exporter for GPU telemetry, processor batching, and dynamic filtering, so every signal is captured and routed reliably
- Automate GitOps-based delivery and continuous verification; embedding quality, integrity, and cost gates into pipelines so releases are policy-compliant by default rather than by manual review
- Close the loop between delivery and observability by using telemetry, evaluation, and cost signals to drive deployment decisions, progressive rollout, and automated rollback of AI workloads
- Ensure cost and telemetry are identity-stamped and per-tenant, so consumption and behavior are attributable end-to-end, keeping FinOps and observability tied to the workloads that generate the load
Skills
- • 8+ years in DevOps, MLOps, platform, or observability engineering, with hands-on production ownership of AI or high-throughput services
- • Strong hands-on DevOps experience, including CI/CD/CV pipelines and GitOps tooling (ArgoCD, Helm, GitHub Actions/GitLab CI, or equivalents) for automated build, test, release, and rollback
- • Hands-on expertise operating inference/model-serving frameworks (Ray Serve, vLLM, Triton, or NIM) on GPU infrastructure
- • Strong experience with observability stacks (Prometheus, Grafana, Loki, Tempo/Jaeger) and OpenTelemetry
- • Experience with API gateways and request routing (Envoy or equivalent), including streaming responses
- • Experience with cost management / FinOps tooling (OpenCost, Kubecost, or equivalent) and quota/rate-limit enforcement
- • Familiarity with model/artifact registries and supply-chain scanning (Harbor, MLflow, Trivy/SBOM)
- • Proven track record operating AI or service infrastructure under compliance, security, or regulatory constraints
- • Ability to define clean ownership boundaries and consumption contracts with platform, trust, and data teams
- • Bachelor's or Master's degree in Computer Science or related technical field
- • Experience with LLM evaluation and debugging tooling (LangSmith, Langfuse) and prompt/response quality measurement
- • Experience with sandboxed/secure execution (gVisor, Firecracker, or microVM isolation) for untrusted or multi-tenant workloads
- • Familiarity with GPU telemetry (DCGM) and GPU utilization optimization
- • Experience with lineage and governance contracts (OpenLineage) and AI license management
- • Exposure to multi-tenant cost attribution and per-tenant SLA/alerting
- • Exposure to regulated delivery environments (financial services, tax, healthcare, risk)
Benefits
- Medical and dental coverage
- Pension and 401(k) plans
- A wide range of paid time off options
- Team-led and leader-enabled hybrid model
- Flexible vacation policy
- EY Paid Holidays
- Winter/Summer breaks
- Personal/Family Care
- Other leaves of absence when needed to support your physical, financial, and emotional well-being
Company Overview
Company H1B Sponsorship