Note: The job is a remote job and is open to candidates in USA. Technology Partners is a technology services company seeking a hands-on Data Engineer to design, test, and maintain data pipelines for advanced analytics. The role focuses on data ingestion, transformation, Snowflake and dbt development, scalable data modeling, production support, and assisting with machine learning and artificial intelligence pipelines.
Responsibilities
- Automate data wrangling from a wide variety of sources (flat files, remote servers, databases, legacy systems, message queues, APIs and more)
- Identify opportunities to improve system and business process efficiencies
- Cleanse/transform data in preparation for all levels of analytical use
- Leverage many technologies and methodologies as required—includes scripting and programming languages
- Assist in the preparation, deployment, and maintenance of machine learning/artificial intelligence data pipelines
- Work with vendors and consultants as appropriate
- Assist with designing scalable data models for real-world applications, including handling slowly changing dimensions (SCDs) and performance tuning
- Applies lean thinking and tools to identify and eliminate waste in all areas of the position
Skills
- Target experience: 5+ years in data engineering with strong recent experience in modern cloud data platforms
- • Bachelor's degree in applicable field
- • 5+ years Data warehousing/analytics, and data engineering
- • CDC Change Data Capture / Data Ingestion — Experience with Qlik Replicate Strong: (may also have similar experience with Fivetran, HVR, Informatica, IBM CDC, Striim, Debezium
- • Snowflake — Strong: 3+ years Production experience with Snowflake, large datasets, performance, and troubleshooting
- • dbt — Strong: Hands-on production experience building and maintaining dbt models, tests, incremental processing, and deployments
- • SQL — Advanced: 5+ years - Strong SQL development, troubleshooting, optimization, and data warehousing skills
- • Production Support: Able to independently troubleshoot pipeline failures, data quality issues, CDC latency, and other production problems
- • 2+ years of experience in object-oriented programming using Python
- • Strong interpersonal skills with professional verbal and written communication abilities
- • Bachelor's degree in Business Analytics, Computer Science, Mathematics, Statistics, or related field
- • Snowflake Data Engineer Certification
- • Ability to work independently as well as within a team environment
- • Effective project management and planning skills
- Direct Qlik experience is preferred but not required
- • Dagster or similar orchestration experience
- • Git / CI/CD
- • DataStage
- • Db2 and/or SQL Server
- • Azure
Company Overview
Company H1B Sponsorship