Note: The job is a remote job and is open to candidates in USA. DataEdge Consulting is seeking a Data Engineering Architect for a global Fortune 500 company in the Food Services industry. The role involves designing and implementing robust data solutions on AWS, collaborating with various teams to ensure the cloud data infrastructure meets organizational needs.
Responsibilities
- Define, build, test, and implement scalable data pipelines
- Design and implement cloud-native data architectures on AWS, including data lakes, data warehouses, and real-time data processing pipelines
- Perform data analysis required to troubleshoot data-related issues and assist in the resolution of data issues
- Collaborate with development, analytics, and reporting teams to develop data models that feed business intelligence tools
- Design and build API integrations to support the needs of analysts and reporting systems
- Develop, deploy, and manage AWS Lambda functions written in Python
- Develop, deploy, and manage AWS Glue jobs written in Python
- Ensure efficient and scalable serverless operations
- Debug and troubleshoot Lambda functions and Glue jobs
- Collaborate with other AWS service teams to design and implement robust solutions
- Optimize data storage, retrieval, and pipeline performance for large-scale distributed systems
- Ensure data security, compliance, and privacy policies are integrated into solutions
- Develop and maintain technical documentation and architecture diagrams
- Stay current with AWS updates and industry trends to continuously evolve the data architecture
- Mentor and provide technical guidance to junior team members and stakeholders
Skills
- Good background on SQL query writing
- Strong background on Python coding
- Data Audit trails
- Proactive communication
- Experience in data architecture, engineering, or similar roles
- Programming with Python
- Experience in ETL or Data Engineering role building and implementing data pipelines
- Strong understanding of design best practices for OLTP systems, ODS reporting needs, and dimensional database practices
- Hands-on experience with AWS Lambda, AWS Glue, and other AWS services
- Proficient in Python and SQL with the ability to write efficient queries
- Experience with API-driven data access (API development experience a plus)
- Solid experience with database technologies (SQL, NoSQL) and data modeling
- Understanding of serverless architecture benefits and challenges
- Experience working in agile development environments
- Highly self-motivated, detail-oriented, and able to work independently
- Strong analytical thinking, problem-solving, and communication skills
- AWS certifications (e.g., AWS Certified Data Analytics - Specialty, AWS Certified Solutions Architect) are highly desirable
- Experience with modern data stack technologies (e.g., dbt, Snowflake, Databricks)
- Familiarity with machine learning pipelines and AI-driven analytics
- Background in DevOps practices and Infrastructure as Code (IaC) using tools like Terraform or AWS CloudFormation
- Knowledge of CI/CD pipelines for data workflows
Benefits
- W2 ONLY
- Remote OK, but Charlotte NC Preferred
Company Overview