Note: The job is a remote job and is open to candidates in USA. NVIDIA’s DGX Cloud organization develops reliable data foundations for fleet health, capacity, utilization, cost, reliability, and operational decision-making across large GPU fleets. The Senior Data and Platform Engineer will build and operate ingestion, transformation, data-quality, security, observability, and self-service systems that turn infrastructure telemetry into dependable governed data products.
Responsibilities
- Own systems end to end. For example, work from ambiguous customer and operational needs through architecture, implementation, deployment, observability, incident response, and ongoing support
- Construct data pipelines and products. Such as designing and maintain batch and streaming ingestion, transformation, reconciliation, and serving paths for fleet, capacity, utilization, cost, scheduling, and operational telemetry
- Build shared libraries, workflow and DAG or equivalent experience abstractions to evolve the data platform. Develop deployment tooling, data contracts, and paved-road patterns that improve team speed and safety
- Engineer reliable distributed workloads. As well as diagnose correctness and performance issues across applications, SQL engines, Spark jobs, storage systems, networks, and cloud services. Build for retries, idempotency, backfills, schema evolution, and partial failure
- Treat security as part of the build. For example, applying least privilege, service identities, secrets management, access controls, environment isolation, auditability, and safe operational practices throughout the system lifecycle
- Improve quality and operations: Establish automated tests, data-quality checks, lineage, freshness and completeness monitoring, actionable alerting, SLOs, and clear ownership
- Deliver consumption experiences. Such as making trusted data usable through well-modeled tables, APIs, automation, dashboards, and focused internal applications—not only through one-off queries
- Raise the engineering bar. Lead build reviews, communicate tradeoffs, mentor other engineers, and improve the team's architecture, testing, debugging, and operational practices
Skills
- BS or MS in Computer Science, Engineering, or a related field, or equivalent experience
- 5+ years of experience building and operating production software, data platforms, backend infrastructure, databases, or distributed systems
- Strong software-engineering fundamentals and production proficiency in Python or another backend or systems language, with the ability and willingness to work primarily in Python and SQL
- Deep hands-on experience in at least one of the following areas: Distributed data processing using Spark or a comparable compute framework, Relational, distributed, or analytical database architecture and operation at scale, Production ETL, change-data-capture, streaming, or event-processing systems, Backend or cloud-platform systems that process, transform, or serve substantial data volumes, Strong SQL and data-modeling skills, including a practical understanding of query performance, schema evolution, incremental processing, consistency, and analytical consumption patterns
- Demonstrated ability to debug unfamiliar systems across multiple layers using logs, metrics, traces, query plans, profiles, and controlled experiments to find root causes
- Experience operating services or pipelines in a cloud or similarly complex production environment, including testing, CI/CD, monitoring, alerting, rollback, and incident response
- Working knowledge of secure platform development, including identity and access management, least privilege, secret handling, trust boundaries, and safe multi-environment deployments
- Ability to make sound architectural tradeoffs, own work through ambiguity, and communicate effectively with users, partner teams, and engineers from different fields
- A track record of learning unfamiliar technologies and domains and turning that learning into maintainable systems and reusable team practices
- Experience with AI agents and LLM-supported workflow automation, particularly as applied to engineering and operational activities
- Experience with Databricks, Apache Spark, PySpark, Spark SQL, Delta Lake, Unity Catalog, or another modern lakehouse or distributed-compute platform
- Experience with Kafka or another streaming platform, change-data capture, event development, partitioning, consumer groups, offset management, or other high-volume event systems
- Experience with scaling, migrating, or performance-tuning relational, distributed, time-series, object-storage, or search-focused data systems, including Elasticsearch or OpenSearch
- Background working with AWS, Azure, GCP, Kubernetes, Slurm, compute clusters, GPU-accelerated infrastructure, or fleet-scale telemetry
- Experience developing agentic systems, LLM-enabled workflow automation, harness engineering, or dependable evaluation and operational tooling for AI agents
Benefits
- Eligible for equity
Company Overview
Company H1B Sponsorship