Note: The job is a remote job and is open to candidates in USA. Tredence Inc. is seeking a Senior Data Engineer to manage end-to-end data engineering delivery and enterprise data platform operations. The role focuses on designing scalable data pipelines, optimizing distributed systems, supporting data migration and quality, and collaborating with stakeholders and clients.
Responsibilities
- Manage end to end delivery by Investigating problem areas, working cross-functionally with product manager & other stakeholders
- Follow the Agile development methodology; think strategically and execute methodically
- Develop and manage capacity and growth projection forecasts of the environment within budgets
- Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional / non-functional business requirements
- Drive optimization, testing and tooling to improve quality of solutions
- Manage teams that build and operate high volume distributed systems in a SaaS environment
- Great at devising efficient processes that increase velocity and quality
- Understanding business use cases and deriving codes as per requirements
- Promote the data discovery, availability and quality capabilities and expertise to business unit leaders and educate them in leveraging these capabilities in achieving their business goals
- Accountable for operational effectiveness in performance, uptime, release management of the enterprise data platform
- Responsible for creating and validating end-to-end scalable data pipeline templates
- Responsible on the migrating the streaming data from the S3 to Databricks layers (raw, processed, consumption) and applying required data quality checks and transformations
- Handle slowly changing dimensions, change data capture, dimensions, facts and manage upsets
- Responsible for handling adhoc data analysis requests
- Responsible for handling client communication
- Responsible for the data accuracy, data governance, file handling, error handling, data validation
Skills
- Master's degree (or foreign equivalent) in Business Analytics, Information Systems, Data Science, or a related field, PLUS one (1) year of experience in the job offered or a related position
- Experience must include demonstrable knowledge of: Data Engineering Pipelines; Data Warehousing; Data Modeling; SQL; Cloud Technology (Google Cloud); Data Wrangling, and; Data Analysis
- Travel to unanticipated client locations throughout the U.S., approximately 30%, as required
- May telecommute from any location within the U.S
Benefits
- May telecommute from any location within the U.S.
Company Overview
Company H1B Sponsorship