Note: The job is a remote job and is open to candidates in USA. Tredence Inc. is seeking a Senior Manager – Data Engineering to lead the architecture and implementation of scalable cloud-based data platforms. The role focuses on data modeling, ETL/ELT pipelines, big data processing, cloud analytics integration, performance optimization, CI/CD, migration support, and data governance and security.
Responsibilities
- Design and architect scalable data platforms in the cloud, ensuring they meet both technical and business requirements
- Lead the technical vision for cloud-based data solutions, aligning architecture and implementation with strategic goals
- Implement and optimize star schema models within cloud data warehouses, focusing on performance and scalability
- Regularly review data workloads to identify opportunities for operational and performance improvements
- Manage capacity in cloud data warehouses by proactively adjusting resources based on usage patterns and demand
- Monitor system performance and ensure optimal operation through tuning and resource management
- Build high-performance data models by applying effective table distribution and indexing strategies
- Implement and manage cloud-based data lakes, leveraging hierarchical namespaces and POSIX-compliant access control mechanisms to ensure secure and organized data storage
- Design, develop, and optimize ETL/ELT pipelines for efficient data movement and transformation
- Implement big data processing solutions to handle large-scale datasets, and automate workflows to support real-time data streaming and analytics
- Integrate cloud-based analytics platforms with other services such as data lakes, orchestration tools, and data warehouses to create seamless data ecosystems
- Manage and maintain cloud compute clusters, optimizing resource utilization for both performance and cost efficiency
- Develop and maintain CI/CD pipelines to support continuous integration and deployment of data processes
- Execute change tasks such as table partitioning using appropriate strategies to enhance query performance and manage data lifecycle effectively
- Identify and resolve performance bottlenecks across the cloud data environment, ensuring systems run efficiently and reliably
- Provide guidance and documentation to developers and end-users on best practices for cloud services, including maintenance and housekeeping
- Address incidents and issues related to data migration from on-premises systems to the cloud, ensuring smooth transitions and minimal disruption
- Uphold data quality, governance, and security by implementing industry best practices and compliance standards
Skills
- Master's degree (or foreign equivalent) in Information Systems, or a related field, PLUS two (2) years of experience in the job offered or a related position
- Experience must include demonstrable knowledge of: SQL; Python; ETL/ELT data pipelines; partitioning tables; Star & Snowflake data models; CI/CD pipelines; AWS S3; Databricks; JIRA, and; Confluence
- Required knowledge may be gained prior to or concurrently with Master's degree
- Travel to unanticipated client locations throughout the U.S., approximately 30%, as required
- May telecommute from any location within the U.S
Benefits
- May telecommute from any location within the U.S.
Company Overview
Company H1B Sponsorship