Note: The job is a remote job and is open to candidates in USA. Upstart is an AI lending marketplace that partners with banks and credit unions to expand access to affordable credit. The Senior Data Platform Engineer owns and improves the Lakehouse platform, focusing on infrastructure, security, compliance, cost efficiency, and reliable tools for data, machine learning, and software engineering teams.
Responsibilities
- Own the Databricks platform across multiple environments, including a newly launched regulated environment (Upstart Bank). Responsibilities include expanding infrastructure as code for new Lakehouse features, enforcing a no-click-ops policy, establishing disaster recovery, implementing least-privilege and attribute-based access controls (e.g., based on data classification), ensuring environment isolation, and resolving other known security risks
- Close control gaps and automate evidence gathering for OCC, SOX and SOC audits across access management, change management, and data quality
- Reduce Databricks and AWS spend through accurate cost attribution, waste detection, and platform-level optimizations (e.g., compute rightsizing, improving pruning, migrating to Graviton and spot instances, Delta Lake Archival)
- Evaluate and adopt new technologies, including open-source lakehouse alternatives, to expand platform capabilities
- Improve developer efficiency and infrastructure reliability by working with cross-functional stakeholders like Data Engineering, Machine Learning, Analytics, Compliance, and Security
- Improve Lakehouse security, usage efficiency, and observability by collaborating closely with the Data Engineering and Security teams
- Evangelize good coding and engineering best practices while uplifting fellow engineers and mentoring junior team members
- Help break down complex projects and requirements into quarterly commitments and sprints
- Own Lakehouse infrastructure components end to end, from technical requirements and design through rollout and production support
Skills
- Bachelor's degree in Computer Science or a related field, or equivalent practical experience
- 5+ years of experience in infrastructure engineering, DevOps, or data platform engineering
- Hands-on experience operating a lakehouse or big data platform (e.g., Databricks, Snowflake, Apache Spark, Delta Lake, Apache Iceberg)
- Hands-on experience with AWS (e.g., IAM, S3, EC2, EKS), Python, infrastructure as code (Terraform or CDK), CI/CD, and Kubernetes
- Experience implementing security and compliance controls in audited environments (e.g., SOX, SOC 2, GDPR, GLBA, OCC), including access management, encryption, and change management
- Experience reducing cloud or data platform costs through usage analysis, cost attribution, and infrastructure optimization (e.g., Graviton or spot instances, storage tiering)
- Ability to reason from first principles and work effectively through ambiguity on complex, cross-functional problems
- Proficient in DevOps engineering, leveraging Continuous Integration/Continuous Delivery (CI/CD) tools such as Kubernetes, Jenkins, CDK, APM, Terraform, and more
- Experienced in automation, alerting, monitoring, security, and declarative infrastructure
- Solid understanding of cloud networking fundamentals
- Skilled at explaining complex concepts in simple, accessible terms and thriving in fast-paced, ambiguous environments
- Experience using AI coding assistants or agents to accelerate infrastructure development and operations
- Familiarity with distributed systems, data architecture design, and big data technologies (e.g., Spark, Kafka, Lakehouse, Databricks)
Benefits
- Target bonuses and equity compensation.
- Annual equity grants that vest quarterly.
- 401(k) or Group Retirement Savings Plan with a company match of $2 for every $1 contributed, up to $15,000 annually (USD in the US, CAD in Canada).
- Employee Stock Purchase Plan (ESPP) with discounted stock purchase options for eligible employees (US only).
- Comprehensive health coverage, including medical, dental, vision, and wellness resources for US employees, and supplemental health coverage for Canada.
- Health Savings Account contributions from Upstart for eligible plans (US only).
- Life insurance and disability coverage.
- Paid time off, sick leave, and company holidays, in line with local requirements.
- Paid family and parental leave; duration varies by country.
- Family-centered benefits to support fertility, parenthood, and caregiving needs.
- Employee Assistance Program (EAP) offering mental health support and life-centered resources.
- Financial wellness resources, including access to financial planning tools and a financial concierge service (US only).
- Annual wellness allowance to support physical and emotional well-being and personal development.
- Annual productivity allowance for relevant tools and resources.
- Team events, all-company updates, and employee resource groups (ERGs).
- Catered lunches and fully stocked micro-kitchens when working from offices in the Bay Area, Austin, Columbus, and New York City.
- The majority of work can be accomplished remotely; most employees can live and work anywhere in the U.S. Regular in-person onsites are encouraged, with cadence varying by team and role; most teams meet once or twice per quarter for 2–4 consecutive days.
Company Overview
Company H1B Sponsorship