Note: The job is a remote job and is open to candidates in USA. Confidential is seeking a skilled Data Engineer to build and optimize scalable data pipelines, architectures, and data sets. The role focuses on data integration, warehousing, processing, automation, monitoring, and performance tuning to support data-driven decision-making.
Responsibilities
- Data Pipeline Development: Design, construct, test, and maintain highly scalable data management systems. Develop and implement architectures that support the extraction, transformation, and loading ETL of data from various sources
- Data Integration: Integrate structured and unstructured data from multiple data sources into a unified data system, ensuring data quality and consistency
- Data Warehousing: Build and maintain data warehouses and data lakes to store and retrieve vast amounts of data efficiently. Optimize the performance of databases and queries to meet business needs
- Data Processing: Implement data processing frameworks e.g., Hadoop, Spark to process large datasets in real-time or batch processing
- Automation and Monitoring: Automate manual processes, optimize data delivery, and develop data monitoring systems to ensure data integrity and accuracy
- Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data needs and provide technical solutions that meet business requirements
- Performance Tuning: Optimize the performance of ETL processes, databases, and data pipelines to handle large volumes of data and reduce processing times
Skills
- Bachelor's degree in computer science, Engineering, Information Technology, or a related field
- 5+ years of experience as a Data Engineer or in a similar role
- Proficiency in Snowflake SQL and experience with relational databases (e.g., MySQL, PostgreSQL)
- Knowledge of data warehousing solutions- BigQuery
- Strong understanding of Matillion ETL tool and processes
- Experience with data modeling and data architecture design
- Good to have hands on programming languages like Python
Benefits
- The option to elect healthcare insurance (Dental, Medical, Vision)
- Major holidays
- Paid sick leave as per state law
Company Overview
Company H1B Sponsorship