Note: The job is a remote job and is open to candidates in USA. The confidential client is seeking a Data Engineer to create enterprise data assets that deliver business value through operational and analytical data. The role designs, builds, and maintains data structures, pipelines, frameworks, and services while collaborating with technical and business stakeholders to support data products, analytics, data quality, and organizational data strategy.
Responsibilities
- Partners with Technology partners to Design, Build, and Maintain the data-based structures and systems in support of Data and Analytics and Data Product use cases
- Builds data pipelines to collect and arrange data and manage data storage in their big data environment
- Builds robust, testable programs for moving, transforming, and loading data using big data tools such as Spark
- Coordinates design and development with Data Products Partners, Data Scientists, Data Management, Data Modelers, and other Technical partners to construct strategic and tactical data stores
- Ensures data is prepared, arranged and ready for each defined business use case
- Designs and deploys frameworks and micro services to serve data assets to data consumers
- Collaborates and aligns with technical and non-technical stakeholders to translate customer needs into Data Design requirements, and work to deliver world-class visualizations, data stories while ensuring data quality and integrity
- Provides consultation to all areas of the organization that plan to use data to make decisions
- Supports any team members in the development of such information delivery and aid in the automation of data products
- Acts as trusted adviser and partner to business leads- assisting in the identification of business needs & data opportunities, understanding key drivers of performance, interpreting business case data drivers, turning data into business value, and participating in the guidance of the overall data and analytics strategy
Skills
- 8 years of professional experience
- Amazon Web Services
- AWS Lambda
- Python Frameworks
- Qlik Replicate
- Snowflake (Platform)
- Build and test data ingestion pipelines - who understands ingestion pipeline engineering, query language, data integration tools like Qlik Replicate or precisely connect, and data quality testing source/target
- Experience with DevOps principals and CI/CD
- Experience with Docker and Kubernetes
- Background in Big Data Engineering and Advanced Data Analytics
- Prior banking or financial Services experience
- Experience developing solutions for the financial services industry
- Experience building data solutions at scale
- Experience designing and building relational data structures in multiple environments
- Experience with Airflow, Argo, Luigi, or similar orchestration tool
- Experience with No-SQL databases such as HBase, Cassandra, or MongoDB
- Experience with streaming technologies such as Kafka, Flink, or Spark Streaming
- Experience working with Hadoop ecosystem building Data Assets at an enterprise scale
- Proven record of accomplishment of delivering operational
Benefits
- Remote work
Company Overview
Company H1B Sponsorship