Note: The job is a remote job and is open to candidates in USA. Data Surge is disrupting the services industry with cutting-edge technology that brings together the very best consultants from the big data, machine learning, and software modernization space. They are seeking a skilled and motivated Data Engineer with deep Databricks and cloud experience to support their Federal Services team by designing and implementing scalable data solutions and optimizing data pipelines.
Responsibilities
- Design, develop, and optimize robust data pipelines in Databricks to process and transform large-scale data sets
- Collaborate with data scientists, engineers, and cloud architects to build reliable, secure data systems
- Automate infrastructure provisioning using Terraform and manage resources on Azure (or AWS/GCP)
- Monitor, troubleshoot, and support production-grade data pipelines and jobs
- Develop CI/CD workflows to streamline deployment and version control of data processing code
- Ensure security, performance, and reliability across all stages of data workflow
- Utilize and manage Docker containers where needed to deploy data services
- Participate in Agile team ceremonies and help shape iterative data delivery plans
- Support continuous improvement in data architecture, automation, and governance practices
- Other duties as reasonably required
Skills
- Due to the nature of the role, this position requires U.S. Citizenship
- 5+ years of experience in data engineering or backend data infrastructure roles
- 2+ years of hands-on experience with Databricks, Spark, or distributed data processing systems in a production environment
- Deep proficiency with PySpark and Python for data engineering tasks
- Experience with data modeling
- Solid experience with Airflow, dbt, or other orchestration and transformation tools
- Proven skills in data governance, lineage, or security best practices
- Experience working with Azure (preferred), AWS, or Google Cloud
- Strong working knowledge of CI/CD tools and version control workflows
- Comfortable working with relational (e.g., PostgreSQL, MySQL) and NoSQL databases
- Proven experience in monitoring, debugging, and scaling cloud-based data systems
- Strong communication skills and ability to work independently in a remote team environment
- Experience with Kubernetes and Helm for deploying data workloads (not required)
- Solid experience with Terraform or other Infrastructure as Code tools
- Familiarity with containerization using Docker
- Knowledge of data lakehouse architectures and Delta Lake
- Experience working on Federal or government projects
- Existing security clearance (preferred but not required)
- Databricks Certification
- Bachelor's (Preferred)
- Data Engineering: 7 years (Preferred)
- Databricks: 3 years (Preferred)
Benefits
- 401(k)
- 401(k) matching
- 401(k) with 4% Match, immediately vested
- Dental insurance
- Health insurance
- Vision insurance
- Paid time off
- PTO – Including holidays + your birthday
- Retirement plan
- Remote-first work environment
- Growth and development opportunities
- Bonuses + more
Company Overview