[Remote] Data Engineer - Databricks

Note: The job is a remote job and is open to candidates in USA. Dice is seeking a Data Engineer with strong Databricks expertise to modernize and scale business intelligence capabilities. The role focuses on building data pipelines, deploying machine learning solutions, operationalizing intelligent analytics, and supporting data architecture, governance, security, and vendor integrations.

Responsibilities

• End-to-end data pipelines and integrations
• Advanced SQL optimization and complex query design
• Kafka streaming applications and connector development
• Databricks workflow development with medallion architecture
• Data governance implementation and compliance
• Performance tuning for large-scale data processing
• Data security and privacy best practices
• Apache NiFi pipeline development for invoice and PO processing
• Integration with purpose-built data stores (Druid, MongoDB, OpenSearch, Postgres)
• Build and maintain end-to-end ML pipelines for training, deployment, and monitoring of models
• Design and optimize data architectures for large-scale ML workloads
• Explore and implement LLM-based solutions, RAG architectures, and generative AI for business use cases
• Cross-functional collaboration with product and engineering teams
• Technical mentoring for junior data engineers
• Analytical thinking for complex data problems
• Stakeholder communication for data requirements
• Process improvement and efficiency focus
• Quality mindset for data accuracy and reliability
• Direct communication with data platform vendors
• Evaluates vendor tools for specific data use cases
• Provides technical feedback on vendor product roadmaps
• Coordinates with vendors for data integration projects

Skills

• Bachelor's/Master's in Computer Science, Data Engineering, Statistics, or related field
• 5+ years in data engineering; 2+ years applying ML in production

Benefits

• The individual may also be eligible for discretionary bonuses.
• Medical, dental, and vision benefits
• The company’s 401(k) retirement plan

Company Overview

Dice offers a marketplace that enables direct connections between recruiters and technologists. It was founded in 1990, and is headquartered in Centennial, Colorado, USA, with a workforce of 201-500 employees. Its website is https://www.uk.dice.com.

Company H1B Sponsorship

Dice has a track record of offering H1B sponsorships, with 2 in 2022, 4 in 2021, 5 in 2020. Please note that this does not guarantee sponsorship for this specific role.