Note: The job is a remote job and is open to candidates in USA. Scout Motors is hiring a Senior DataOps Engineer to build and operate enterprise-scale cloud data platforms and data pipelines. The role also covers data integration and modeling, BI dashboards, machine learning, infrastructure reliability, monitoring, and data governance.
Responsibilities
- Contribute to the design, implementation, automation, maintenance, and operational support of enterprise-scale cloud data platforms leveraging Databricks, AWS cloud services, and modern Infrastructure as Code (IaC) practices
- Lead the design, development, optimization, and operational management of enterprise-scale ETL/ELT pipelines within Databricks
- Build and maintain scalable batch and streaming data pipelines using PySpark, Spark SQL, Delta Lake, and Databricks Workflows
- Configure, optimize, and manage Databricks clusters for performance, scalability, reliability, and cost efficiency
- Work closely with software development and systems teams to build Data Integration solutions
- Design and build Data models using tools such as Lucid, Talend, Erwin, MySQL workbench
- Define and enhance enterprise data model to reflect relationships and dependencies
- Review application data systems to ensure adherence to data governance policies
- Design and build ETL (Python), ELT(Python) infrastructure, automation, and solutions to transform data as required
- Design and Implement BI dashboards to visualize Trends and Forecasts
- Design and implement data infrastructure components, ensuring high availability, reliability, scalability, and performance
- Design, train and deploy ML models
- Implement monitoring solutions to proactively identify and address potential issues
- Collaborate with security teams to ensure the data platform meets industry standards and compliance requirements
- Collaborate with cross-functional teams, including product managers, developers, and business partners to ensure robust and reliable systems
Skills
- Bachelor's degree in computer science, information technology, or related field or equivalent work experience
- 8+ years of hands-on experience as DataOps Engineer in a manufacturing or automotive environment
- Experience with streaming and event-based architecture
- Experience implementing data lakehouse solutions using Databricks
- Experience with infrastructure as code (Terraform)
- Proficient in building data pipelines using languages such as Python and SQL
- Experience with AWS based data services such as Glue, Kinesis, Firehose or other comparable services
- Experience with Structured, unstructured and time series databases
- Solid understanding of cloud data storage solutions such as RDS, DynamoDB, DocumentDB, Mongo, Cassandra, Influx
- Several years of experience working with cloud platforms such as AWS and Azure
- Proven ability to develop and deploy scalable ML models
- Hands-on experience in designing, training, and deploying ML models
- Strong ability to extract actionable insights using ML techniques
- Ability to leverage ML algorithms for forecasting trends and decision-making
- Excellent problem-solving and troubleshooting skills
- Effective communication and collaboration skills
Benefits
- Annual performance bonus program
- Remote role based within the United States; not eligible for remote work in New York City
- Competitive insurance including medical, dental, vision and income protection plans
- 401(k) program with an employer match and immediate vesting
- 20 days planned PTO, as accrued
- 40 hours of unplanned PTO and 14 company or floating holidays, annually
- Up to 16 weeks of paid parental leave for biological and adoptive parents of all genders
- Paid leave for circumstances related to bereavement, jury duty, voting time, or military leave
Company Overview
Company H1B Sponsorship