Note: The job is a remote job and is open to candidates in USA. TechnoidLLC is seeking a Senior Azure Data Engineer to develop and maintain high-performance cloud data platforms supporting business intelligence, analytics, and reporting. The role focuses on building scalable data pipelines, ETL/ELT workflows, data lake and lakehouse solutions, data models, quality frameworks, and automated deployments while collaborating with architects, stakeholders, and cross-functional teams.
Responsibilities
- Design, develop, and maintain scalable and high-performance data pipelines using Azure Data Factory and Azure Databricks
- Build complex ETL/ELT workflows for data ingestion, transformation, integration, and processing from multiple source systems
- Develop advanced data processing solutions using Python, PySpark, Apache Spark, and SQL
- Design and implement data lake and lakehouse solutions using Azure Data Lake Storage Gen2 and Delta Lake
- Develop and optimize data models, data transformations, and cloud data warehouse solutions
- Implement incremental data loads, change data capture (CDC), and efficient data processing strategies
- Optimize Spark jobs, SQL queries, pipeline execution, and cloud resource utilization
- Establish data quality checks, validation frameworks, monitoring, logging, and error-handling mechanisms
- Implement CI/CD pipelines, source control, and automated deployment processes using Azure DevOps and Git
- Ensure data security, access controls, governance, and compliance with enterprise standards
- Collaborate with data architects, application teams, business analysts, and stakeholders to translate business requirements into technical solutions
- Provide technical guidance, conduct code reviews, troubleshoot production issues, and mentor junior data engineers
- Contribute to data platform modernization, cloud migration, and continuous improvement initiatives
Skills
- US Citizen / Green Card
- 12+ years of experience
- 12+ years of overall IT experience, with significant hands-on experience in Data Engineering and enterprise data solutions
- Strong expertise in SQL, Python, and data transformation techniques
- Extensive hands-on experience with Azure Data Factory (ADF)
- Strong experience with Azure Databricks, PySpark, and Apache Spark
- Experience with Azure Data Lake Storage Gen2 (ADLS Gen2)
- Strong understanding of ETL/ELT architecture, data warehousing, data modeling, and data integration
- Experience designing, developing, and supporting production-grade data pipelines
- Knowledge of Delta Lake, incremental processing, and distributed data processing
- Experience with Azure DevOps, Git, CI/CD, and deployment automation
- Strong problem-solving, analytical, communication, and stakeholder collaboration skills
- Ability to work independently, own technical deliverables, and support enterprise-scale data engineering initiatives
- Green Card
Company Overview