Note: The job is a remote job and is open to candidates in USA. firstPRO, Inc is seeking a Senior Data Engineer for a long-term contract supporting an established cloud-based data platform. The role focuses on maintaining, troubleshooting, and optimizing enterprise data pipelines, while also developing scalable transformations, integrations, and enhancements in an Azure lakehouse environment.
Responsibilities
- Maintain, troubleshoot, optimize, and enhance existing enterprise data pipelines
- Develop scalable data pipelines and transformations using Python, PySpark/Spark, and Databricks
- Ingest and transform data across raw, transformed, and curated lakehouse layers
- Monitor pipeline performance, reliability, dependencies, and resource consumption
- Identify inefficient workloads and implement improvements to processing performance and cloud costs
- Develop smaller new pipelines, enhancements, and data integrations as needed
- Collaborate with data architects, engineers, reporting teams, and offshore development/support resources
- Produce clear technical documentation covering code, pipelines, dependencies, processes, and technical decisions
- Support an established production data environment while following existing architecture and development standards
Skills
- 7+ years of Data Engineering experience, ideally at the Senior Data Engineer level
- Strong, hands-on production experience with PySpark or Apache Spark
- Strong hands-on development experience with Databricks
- Advanced Python development skills
- Demonstrated experience building, maintaining, troubleshooting, and optimizing enterprise data pipelines
- Strong understanding of distributed data processing and lakehouse/data-lake architectures
- Experience working within an established production codebase and data platform
- Strong technical documentation and communication skills
- Ability to work independently while collaborating effectively across technical teams
- Must be based in the U.S. and available to work Eastern Time business hours
- Azure and Azure Data Factory experience preferred
- Familiarity with Microsoft Fabric is a plus
Benefits
- Fully remote within the U.S.
Company Overview