Note: The job is a remote job and is open to candidates in USA. Abacus is seeking a Data Engineer to develop, maintain, and support data pipelines within a modern Azure-based data platform. The role focuses on Python and SQL Server ETL development, data integration, pipeline monitoring, troubleshooting, documentation, and collaboration in regulated environments.
Responsibilities
- Develop and maintain ETL pipelines using Python and SQL Server for data ingestion, transformation, and integration from various data sources, including structured files and RESTful APIs
- Write, test, and debug Python code for data processing, automation, and basic integrations following established standards and best practices
- Create and maintain T-SQL queries, views, and stored procedures to support business logic and reporting requirements
- Assist in building and operating data workflows using Azure Data Factory, Azure SQL, and Azure Blob Storage
- Support the monitoring of data pipelines and help troubleshoot data quality issues, pipeline failures, and performance problems
- Follow defined data quality, security, and compliance procedures in regulated environments such as healthcare and financial services
- Collaborate with data analysts, software engineers, and DevOps teams to understand data requirements and upstream systems
- Participate in code reviews, implement feedback, and continuously improve coding and engineering practices
- Create and maintain clear documentation for data pipelines, logic, and operational processes
Skills
- 1–4 years of professional experience in Data Engineering, Software Engineering, or a related technical role
- Strong hands-on proficiency in Python, including writing functions, handling errors, and debugging code
- Experience working with relational databases, preferably SQL Server
- Strong working knowledge of SQL, including joins, aggregations, subqueries, and basic performance considerations
- Exposure to integrating or consuming data from RESTful APIs or external data sources
- Familiarity with Git or similar version control systems
- Strong analytical and problem-solving skills with the ability to learn quickly
- Exposure to Azure data services such as Azure Data Factory, Azure SQL, or Blob Storage
- Basic understanding of ETL concepts, data validation, and pipeline monitoring
- Familiarity with data modeling fundamentals, including tables, keys, and relationships
- Awareness of data security, privacy, and compliance standards such as HIPAA and SOC2
- Ability to design and support scalable ETL pipelines using Python and SQL Server for data ingestion, transformation, and integration from diverse sources
- Ability to develop and optimize T-SQL stored procedures to support business logic and reporting needs
- Ability to support secure and efficient data workflows using Azure Data Factory, Azure SQL, and Azure Functions
- Ability to help ensure data quality, lineage, and compliance requirements are maintained
- Ability to collaborate with software engineering, data analytics, security, and DevOps teams
- Ability to monitor and troubleshoot pipeline failures and data discrepancies
- Ability to participate in code reviews and continuously improve engineering practices
- Ability to work effectively in a collaborative, fast-paced environment
- Good written and verbal communication skills
- Bachelor's Degree
- Experience with additional Azure services such as Azure Synapse, Azure DevOps, and Azure Functions
- Understanding of data modeling and warehousing concepts
- Exposure to cybersecurity data, SIEM tools, or SOC operations
- Knowledge of .NET Framework and C#-based APIs, particularly in data consumption contexts
- Background in MSP/MSSP environments or consulting
- Familiarity with Power BI or other data visualization tools
- Experience with Power BI or other data visualization tools
- Basic understanding of application systems or APIs built using .NET or similar frameworks
Benefits
- Remote in US
- Opportunities for hands-on learning and growth under the guidance of senior data engineers
Company Overview
Company H1B Sponsorship