Note: The job is a remote job and is open to candidates in USA. Intellisoft Technologies is seeking a Data Engineer to join their team. The role involves designing, developing, and maintaining scalable ETL pipelines while ensuring data quality and availability, as well as implementing monitoring solutions for data pipeline reliability.
Responsibilities
- Design, develop, and maintain scalable ETL pipelines to ensure data quality and availability
- Implement monitoring and alerting solutions to ensure data pipeline reliability and performance
- Develop and manage deployment pipelines to facilitate continuous integration and delivery of data engineering solutions
- Document and communicate data engineering processes and standards to business-intelligence, data, and analytics professionals with varied backgrounds
- Continuously evaluate and improve data engineering tools and approaches to enhance performance and efficiency
Skills
- Candidates must be very strong as a Data Engineer - Design, develop, and maintain scalable ETL pipelines
- Must be strong in Python and Scala and have work with Big Data Tools
- Very good communication
- Expertise in data engineering languages such as Scala (preferred) or Java, with proficiency in Python
- Experience with BigData tools, particularly Spark, and familiarity with Hadoop ecosystems
- Proficiency in building and managing ETL pipelines
- Experience with cloud platforms like AWS, including services such as S3, Lambda, Redshift, and EMR
- Strong understanding of relational databases and SQL, and familiarity with NoSQL databases
- Knowledge of data architecture, data warehousing, and data marts
- Experience in implementing monitoring solutions and deployment pipelines for data applications
- Ability to work with distributed systems and manage data storage solutions
- Demonstrated ability and desire to continually expand skill set, and learn from and teach others
Company Overview