Note: The job is a remote job and is open to candidates in USA. Tredence Inc. is a data and analytics company seeking a Senior Data Engineer to lead end-to-end delivery and develop scalable enterprise data platforms. The role involves designing and optimizing data pipelines, managing distributed systems and data migrations, ensuring data quality and governance, supporting analytics, and collaborating with stakeholders and clients.
Responsibilities
- Manage end to end delivery by Investigating problem areas, working cross-functionally with product manager & other stakeholders
- Follow the Agile development methodology; think strategically and execute methodically
- Develop and manage capacity and growth projection forecasts of the environment within budgets
- Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional / non-functional business requirements
- Drive optimization, testing and tooling to improve quality of solutions
- Manage teams that build and operate high volume distributed systems in a SaaS environment
- Great at devising efficient processes that increase velocity and quality
- Understanding business use cases and deriving codes as per requirements
- Promote the data discovery, availability and quality capabilities and expertise to business unit leaders and educate them in leveraging these capabilities in achieving their business goals
- Accountable for operational effectiveness in performance, uptime, release management of the enterprise data platform
- Responsible for creating and validating end-to-end scalable data pipeline templates
- Responsible on the migrating the streaming data from the S3 to Databricks layers (raw, processed, consumption) and applying required data quality checks and transformations
- Handle slowly changing dimensions, change data capture, dimensions, facts and manage upsets
- Responsible for handling adhoc data analysis requests
- Responsible for handling client communication
- Responsible for the data accuracy, data governance, file handling, error handling, data validation
Skills
- Master's degree (or foreign equivalent) in Business Analytics, Information Systems, Data Science, or a related field
- One (1) year of experience in the job offered or a related position
- Demonstrable knowledge of Data Engineering Pipelines
- Demonstrable knowledge of Data Warehousing
- Demonstrable knowledge of Data Modeling
- Demonstrable knowledge of SQL
- Demonstrable knowledge of Cloud Technology (Google Cloud)
- Demonstrable knowledge of Data Wrangling
- Demonstrable knowledge of Data Analysis
Benefits
- May telecommute from any location within the U.S.
Company Overview
Company H1B Sponsorship