Note: The job is a remote job and is open to candidates in USA. Mount Sinai Health System is an academic medical and healthcare organization committed to advancing clinical care, education, and research. The Data Engineer II develops and maintains data collection, storage, transformation, and pipeline solutions, supporting big data, streaming, research, and clinical analytics initiatives.
Responsibilities
- Facilitates data collection from a variety of different sources, getting it in the right formats, assuring that it adheres to data quality standards, and assuring that downstream users can get that data quickly and with a common standard interface
- Ensures that data streams/pipelines are scalable, repeatable, and secure, and can serve multiple users within the Institute
- Develops as a core member of an Agile team, using Agile tools and methodology. Work closely with other team members including Application Developers, Database Developers, and Data Scientists
- Responsible for creating the infrastructure that provides insight from raw data and handles diverse sources of data seamlessly
- Enables big data and batch/real-time analytical solutions that leverage emerging technologies
- Additional responsibilities include developing prototypes and proof of concepts for the selected solutions, and implementing complex big data projects with a focus on collecting, parsing, and managing large sets of data using multiple platforms to allow for Research and Data Science initiatives
- Translates business requirements into modern data pipeline solutions. Create centralized documents and diagrams of all solutions
- Creates a data catalog store of all metadata
- Designs and implements monitoring, backup, and disaster recovery of data systems
- Approaches all relationships with a world-class customer service approach. Maintains a customer-focused approach with users to provide solutions that are science/research-driven
- Responsible for the integrity and security of data in all forms of storage throughout the Data Architecture
- Works with other IT professionals through Mount Sinai effectively. Comply with the Institutional Review Board and HIPAA to follow all applicable policies and procedures
- Assists in the development of standards and procedures affecting data management, design and maintenance.Documents all standards and procedures
- Provides presentations and training to other team members in the above
- Possesses an extremely flexible attitude. Willing to work with multiple types of technologies and languages with an open mind and without technology bias. Continuous interest in updating skill sets and knowledge of trends in the Big Data Technology space
- Other duties as assigned
Skills
- * Bachelors degree in Computer Science or a related discipline
- * 4+ years relevant professional development experience
- * In depth knowledge of the Azure cloud platform including Azure Data Factory and Databricks
- * Strong SQL and NoSQL Database Knowledge: Oracle, PostgreSQL/MYSQL, and Mongo DB (or similar)
- * Proficiency on installation and configuration of big data software and technology
- * Knowledge of Hadoop, Spark, Kafka and other big data technology stacks and streaming tools
- * Knowledge of healthcare data, HL7, and Mirth
- * Experience with CI/CD and Observability tools
- Also, experience with version control (Git)
- * Strong skills in data structures, data/file formats, algorithms and object oriented design
- * Proficiency with at least 2 programming languages among Scala/Python/ Java
- Must be flexible and fast to pick up new languages
- * Proficiency in Restful service development
- * Experience with micro-services and SOA
- * Proficiency on installation and configuration of big data software and technology
- * Knowledge of Hadoop, Spark, Kafka and other big data technology stacks and streaming tools
- * Familiarity with and the ability to leverage a wide variety of open source technologies and tools
- * Experience working in an Agile methodology
- Advanced degree preferred
- Preferably in a Data Engineering role
- * Knowledge of healthcare data, HL7, and Mirth are also a big plus
- Preferably with Node JS, Django and PHP
- Working knowledge of cloud architecture and implementation on Azure or AWS, is a big plus
- Experience with serverless computing, creating VMs, cloud security, and other cloud services is also a big plus
- Experience working with JIRA is a plus
Benefits
- This is a remote/hybrid position
Company Overview
Company H1B Sponsorship