Note: The job is a remote job and is open to candidates in USA. Tredence Inc. is hiring a Senior Manager – Data Engineering to design, architect, and lead cloud-based data platforms and solutions. The role focuses on data warehousing and modeling, ETL/ELT and big data pipelines, cloud infrastructure optimization, CI/CD, and data quality, governance, and security.
Responsibilities
- Design and architect scalable data platforms in the cloud, ensuring they meet both technical and business requirements
- Lead the technical vision for cloud-based data solutions, aligning architecture and implementation with strategic goals
- Implement and optimize star schema models within cloud data warehouses, focusing on performance and scalability
- Regularly review data workloads to identify opportunities for operational and performance improvements
- Manage capacity in cloud data warehouses by proactively adjusting resources based on usage patterns and demand
- Monitor system performance and ensure optimal operation through tuning and resource management
- Build high-performance data models by applying effective table distribution and indexing strategies
- Implement and manage cloud-based data lakes, leveraging hierarchical namespaces and POSIX-compliant access control mechanisms to ensure secure and organized data storage
- Design, develop, and optimize ETL/ELT pipelines for efficient data movement and transformation
- Implement big data processing solutions to handle large-scale datasets, and automate workflows to support real-time data streaming and analytics
- Integrate cloud-based analytics platforms with other services such as data lakes, orchestration tools, and data warehouses to create seamless data ecosystems
- Manage and maintain cloud compute clusters, optimizing resource utilization for both performance and cost efficiency
- Develop and maintain CI/CD pipelines to support continuous integration and deployment of data processes
- Execute change tasks such as table partitioning using appropriate strategies to enhance query performance and manage data lifecycle effectively
- Identify and resolve performance bottlenecks across the cloud data environment, ensuring systems run efficiently and reliably
- Provide guidance and documentation to developers and end-users on best practices for cloud services, including maintenance and housekeeping
- Address incidents and issues related to data migration from on-premises systems to the cloud, ensuring smooth transitions and minimal disruption
- Uphold data quality, governance, and security by implementing industry best practices and compliance standards
Skills
- Master's degree (or foreign equivalent) in Information Systems, or a related field, PLUS two (2) years of experience in the job offered or a related position
- SQL
- Python
- ETL/ELT data pipelines
- Partitioning tables
- Star & Snowflake data models
- CI/CD pipelines
- AWS S3
- Databricks
- JIRA
- Confluence
Benefits
- May telecommute from any location within the U.S.
Company Overview
Company H1B Sponsorship