Note: The job is a remote job and is open to candidates in USA. Smart IT Frame LLC is seeking a Lead Data Engineer to build and maintain scalable data platforms. The role focuses on developing ETL/ELT pipelines, real-time streaming solutions, data models, and integrations while ensuring data quality, security, performance, reliability, and scalability.
Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using AWS Glue
- Build and manage real-time data streaming solutions using Apache Kafka
- Develop and optimize data models, schemas, and queries in PostgreSQL
- Integrate data from multiple sources into centralized data platforms and data lakes
- Monitor and troubleshoot ETL jobs, streaming applications, and database performance issues
- Collaborate with business stakeholders, analysts, and data scientists to understand data requirements
- Implement data quality, governance, and security best practices
- Perform performance tuning and optimization of Glue jobs, Kafka topics, and PostgreSQL databases
- Automate deployment and operational processes using DevOps and Infrastructure-as-Code practices
- Ensure high availability, reliability, and scalability of data platforms
Benefits
- Remote work
Company Overview
Company H1B Sponsorship