Note: The job is a remote job and is open to candidates in USA. ECOM is a company focused on digital transformation in the agriculture industry. They are seeking a Data Engineer to build a data lake using AWS stack, integrate data sources, and support analytics projects in a collaborative team environment.
Responsibilities
- Develop and deploy large scale advanced analytics applications into a production environment
- Collaboratively develop code in object-oriented language for a production environment
- Work in Linux environment (bash scripting)
- Utilize SQL for Data Normalization
- Implement Python for data processing
- Understand and apply Data Structures (arrays, lists, stacks, queues, trees, dictionaries, etc.)
- Employ data mining techniques and methodologies (data prep/modeling, classification, regression, clustering, causal modeling, AI, machine learning, ensemble approaches)
- Create effective data visualizations using advanced data visualization tools
- Rapidly develop proof of concepts and test new ideas, scaling them into production ready models
- Leverage cloud platforms such as Amazon Web Services (AWS), Google Cloud Platform (GCP) or Microsoft Azure
- Utilize AWS big data services (Kinesis, Glue, EMR, Spectrum etc.)
- Work with ETL & analytics tools (AWS, Matillion, Tallend, AtScale, etc.)
- Implement data virtualization technologies (Denodo, etc.)
- Utilize data visualization tools (PowerBI, Tableau, etc.)
- Participate in Scrum/Agile delivery processes and DevOps concepts (i.e. CI/CD)
- Work with data storage technologies (Redshift, S3, Hadoop, Graph, etc.)
- Use multiple languages and syntax (Python, SQL, Apache Spark, etc.)
- Manage Container based orchestration of big data pipelines
- Implement Rest APIs for pulling or pushing data to/from internal and external applications
Skills
- A minimum of three years developing and deploying large scale advanced analytics applications into a production environment
- Minimum of five years, hands-on experience collaboratively developing code in object-oriented language for a production environment, including:
- Experience working in Linux environment (bash scripting)
- SQL (Data Normalization)
- Python
- Data Structures (arrays, lists, stacks, queues, trees, dictionaries, etc.)
- Experience in data mining techniques and methodologies (data prep/modeling, classification, regression, clustering, causal modeling, AI, machine learning, ensemble approaches)
- Advanced experience in data visualization tools with a strong grasp of effective data modeling and visualization practices
- Experience rapidly developing proof of concepts and testing new ideas, as well as scale these ideas into production ready models
- Experience leveraging cloud platforms such as Amazon Web Services (AWS), Google Cloud Platform (GCP) or Microsoft Azure
- Advanced, relevant education, advanced Degree (Masters or PhD) in engineering, mathematics, physics, economics, computer science, statistics, or business analytics
- Experience with AWS big data services (Kinesis, Glue, EMR, Spectrum etc.)
- Experience with ETL & analytics tools (AWS, Matillion, Tallend, AtScale, etc.)
- Experience with data virtualization technologies (Denodo, etc.)
- Experience with data visualization tools (PowerBI, Tableau, etc.)
- Experience with Scrum/Agile delivery processes and DevOps concepts (i.e. CI/CD)
- Experience with data storage technologies (Redshift, S3, Hadoop, Graph, etc.)
- Experience with cloud Platforms (AWS, Azure, Google)
- Experience with multiple languages and syntax (Python, SQL, Apache Spark, etc.)
- Experience with Container based orchestration of big data pipelines
Company Overview
Company H1B Sponsorship