Note: The job is a remote job and is open to candidates in USA. Dice is recruiting for a Data Engineer to support a direct client in the healthcare and pharmaceutical domain. The role focuses on designing, developing, and maintaining scalable ETL/ELT data pipelines, managing Databricks data architecture, improving data quality and governance, and supporting analytical stakeholders.
Responsibilities
- Design and develop ETL/ELT data pipelines
- Ensure data pipelines are scalable, reliable, and efficient, meeting the needs of the organization
- Implement and manage the data architecture, including Landing Zone, Bronze, Silver, and Gold layers within the Databricks environment
- Develop processes for data cleaning, normalization, and enrichment to maintain high data quality standards
- Integrate core data management with existing systems to ensure seamless data flow and consistency across platforms
- Develop a comprehensive tagging system for core data, ensuring that all data assets are tagged appropriately for easy retrieval and association
- Monitor and optimize the performance of data pipelines and processing workflows
- Ensure data architecture supports different stages of the data lifecycle, from development to production, with appropriate access controls
- Work closely with data scientists, data stewards, and other stakeholders to understand their data requirements and support their analytical needs
- Provide technical support and guidance to other team members and departments as needed
Skills
- Proven experience as a Data Engineer or in a similar role, with previous experience in the pharmaceutical industry being highly regarded
- Hands-on experience with AWS, scripting and ETL pipelines
- Strong proficiency in programming languages such as Python
- Jira exp
- Strong analytical and problem-solving skills
- Excellent understanding of ETL/ELT processes and data pipeline development
- Knowledge of metadata management and data quality processes
- Proficiency in data modeling, data warehousing, and big data processing frameworks
- Excellent communication and collaboration skills
- Ability to learn new technologies quickly and develop new skills as needed
- Capable of identifying the best approach forward for various data engineering challenges
- Ability to work in a fast-paced, dynamic environment and manage multiple priorities
Benefits
- Remote work arrangement
Company Overview
Company H1B Sponsorship