Senior Software Data Engineer, CCS
This position is remote
We are more than just a healthcare company; we are a community of professionals driven by the knowledge that our success leads to a positive impact on the communities that we support.
At ComplexCare Solutions, you’ll play a crucial part in innovating solutions that empower health plan members and healthcare providers alike. Our collaborative environment fosters creativity and initiative, ensuring that every idea and effort contributes to meaningful change in the industry. Join us and become a part of a team that turns vision into action, making a tangible impact on the future of healthcare.
Overview: The Senior Data Engineer is responsible for contributing to data analytics, data operations, and software product development across the organization. They will support existing data operations such as data pipeline, ETL, data lake / warehouse, data structure development, and troubleshooting of data issues. They will be part of the team which is building next generation data & reporting platform. This platform will cater internal business stakeholders and external customers to provide insights & forecasting to understand current state of business, improve decision-making for their tactical and strategic goals & KPIs. They will work across a variety of products and projects as needed and engage with a variety of data types. This position may require independent work, sharing information and assisting others with work request.
Duties and Responsibilities:
- Work with agile teams and actively participate in sprint planning, backlog grooming, daily standups, retrospectives, and demonstrations.
- Contribute to requirements analysis, technical design discussions, effort estimation, task management, and status reporting.
- Design, develop, and support scalable cloud-native data solutions utilizing Azure Databricks, Delta Lake, Azure Data Factory (ADF), Azure Storage, and Azure SQL technologies.
- Develop and optimize PySpark, Spark SQL, SQL, and Databricks Notebook-based data pipelines for batch and near real-time processing.
- Design and implement Medallion Architecture (Bronze, Silver, Gold layers) and Delta Lake solutions to support enterprise data platforms.
- Build and maintain scalable ETL/ELT frameworks for ingesting, validating, transforming, cleansing, and loading large volumes of structured and unstructured data.
- Design and develop reusable data ingestion and processing frameworks that support configurable business rules, validations, exception handling, and auditability.
- Partner with data architects, business analysts, and application teams to design logical and physical data models supporting enterprise analytics and reporting.
- Design and implement STAR and SNOWFLAKE schemas, dimensional models, and curated data marts optimized for reporting and analytics workloads.
- Develop and maintain data quality, reconciliation, monitoring, lineage, and operational support frameworks to ensure data integrity and reliability.
- Design and develop scalable data export solutions for downstream applications, reporting platforms, and external customer integrations.
- Optimize Databricks workloads, Spark jobs, Delta tables, and SQL queries to improve performance, scalability, and cost efficiency.
- Implement and support CI/CD processes, Infrastructure as Code (IaC), deployment automation, and DataOps best practices across the data platform.
- Collaborate with Power BI and Analytics teams to optimize semantic models, reportingdatasets, and query performance.
- Participate in troubleshooting production issues, root cause analysis, incident resolution, and operational support activities.
- Produce technical design documentation, architecture diagrams, data flow diagrams, operational runbooks, and support documentation.
- Maintain compliance with ComplexCare Solutions (CCS) policies, procedures, security standards, and mission statement.
- Adhere to all confidentiality and HIPAA requirements as outlined within CCS policies and procedures with respect to PHI and sensitive healthcare data.
- Fulfill additional responsibilities as assigned to support operational excellence and organizational objectives.
Job Requirements:
- Minimum 7+ years of experience in Data Engineering, Data Warehousing, ETL/ELT development, and enterprise data platforms.
- 5+ years of hands-on experience with Azure Databricks, PySpark, Spark SQL, Delta Lake, and distributed data processing frameworks.
- 5+ years of experience building and supporting enterprise-scale data pipelines and cloud-native data platforms.
- Strong experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure SQL Database, and related Azure services.
- Strong SQL development skills including query optimization, performance tuning, indexing strategies, and large-scale data processing.
- Experience implementing Medallion Architecture, Delta Lake, Data Lakehouse, and modern analytics platforms.
- Experience with CI/CD, Git, Azure DevOps, Infrastructure as Code, and DataOps practices.
- Experience designing dimensional models, STAR schemas, SNOWFLAKE schemas, and enterprise data warehouses.
- Experience building data quality frameworks, reconciliation processes, monitoring, alerting, and operational support solutions.
- Experience working with reporting and analytics platforms such as Power BI, Tableau, or similar tools.
- Strong understanding of cloud architecture, data governance, security, and role-based access controls.
- Healthcare industry experience and familiarity with HIPAA, PHI, claims, clinical, or member data is highly preferred.
- Excellent problem-solving, analytical, and troubleshooting skills.
- Strong verbal and written communication skills with the ability to work effectively across technical and business teams.
- Ability to work independently, manage competing priorities, and deliver solutions in a fast-paced environment.
Education:
- BS degree in Computer Science or Computer Engineering, Business, or equivalent experience.
- Some travel (less than 10%) may be required for this position, primarily for training and collaboration purposes.
Compensation: $129,400 - $135,000 annually
Annual Bonus Target: 5%
The Company is proud to be an equal opportunity workplace and is an affirmative action employer. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, or any other protected characteristic under federal, state, or other applicable laws.
This company utilizes E-Verify.
7+ years data engineering experience, 5+ years Azure Databricks/PySpark/Spark SQL/Delta Lake, strong SQL, Azure services, CI/CD, Git, Azure DevOps, dimensional modeling, data quality frameworks, healthcare experience preferred, BS in CS/CE/Business or equivalent.
Key Responsibilities
- designing data pipelines
- developing cloud solutions
- optimizing data performance
Skills & Tools
Azure Databricks, Delta Lake, Azure Data Factory, Azure Storage, Azure SQL, PySpark, Spark SQL, SQL, Databricks Notebook, Power BI, Tableau, Git, Azure DevOps
Job Details
- Category: Software Development
- Seniority: Senior Level
- Commitment: Full Time
- Workplace: Remote — United States
- Salary: USD 129,400 – 135,000 / year
- Languages: English
About ComplexCare Solutions
A healthcare company innovating solutions that empower health plan members and healthcare providers. — Industry: Healthcare