Note: The job is a remote job and is open to candidates in USA. Amerit Fleet Solutions is seeking an experienced Data Engineer to establish and maintain a robust data lake infrastructure for enterprise analytics, reporting, and AI/ML initiatives. The role focuses on data architecture, integration, quality, governance, security, performance, documentation, and collaboration across IT, analytics, operations, and AI teams.
Responsibilities
- **Data Lake Architecture & Design:** Design and implement a scalable, cloud-based data lake architecture (AWS/Azure/GCP) that ingests, stores, and manages petabyte-scale data from fleet management systems, maintenance records, vendor systems, and operational databases. Establish data zones (raw, curated, analytics) with appropriate access controls and retention policies
- **Data Integration & ETL/ELT Pipelines:** Build and maintain automated data pipelines that extract, transform, and load data from multiple sources (work order systems, telematics platforms, financial systems, CRM) into the data lake. Ensure real-time and batch processing capabilities with minimal latency. Document all transformations and business logic
- **Data Quality & Integrity Management:** Establish and implement comprehensive data quality frameworks including validation rules, anomaly detection, and reconciliation processes. Monitor data accuracy, completeness, and consistency. Create data quality dashboards and alerts to identify and remediate data issues before they impact downstream analytics. Maintain detailed audit trails for all data changes
- **Data Governance & Compliance:** Develop and enforce data governance policies including data cataloging, metadata management, lineage tracking, and PII/sensitive data protection. Ensure compliance with data privacy regulations (GDPR, CCPA, etc.). Establish data access controls, role-based permissions, and audit logging. Maintain data dictionary and documentation standards
- **Performance Optimization & Monitoring:** Monitor data lake performance, query execution times, and storage utilization. Optimize data structures, indexing, and partitioning strategies to ensure sub-second query response times. Implement automated scaling policies and cost optimization initiatives. Provide recommendations for infrastructure improvements
- **Data Security & Disaster Recovery:** Implement encryption, secure data access protocols, and disaster recovery/business continuity plans. Establish backup, replication, and recovery procedures with defined RPO/RTO targets. Conduct security audits and vulnerability assessments. Maintain compliance documentation for SOC 2 and other security standards
- **Documentation & Knowledge Transfer:** Create comprehensive technical documentation for data lake architecture, data flows, transformation logic, and operational procedures. Develop runbooks for common operations and troubleshooting. Provide training to analysts, data scientists, and other teams on data access, usage best practices, and available datasets
- **Cross-Functional Collaboration:** Partner with business units to understand data requirements and use cases. Collaborate with AI/ML teams on model training data pipelines. Work with analytics teams to optimize queries and reporting. Support data strategy discussions and roadmap planning
Skills
- * Advanced SQL and relational database design (PostgreSQL, MySQL, or SQL Server)
- * Cloud data platforms (AWS S3, Glue, Redshift, Azure Data Lake Gen2, Synapse, Databricks, Microsoft Fabric)
- * ETL/ELT tools (Apache Airflow, dbt, Talend, or cloud-native alternatives)
- * Data warehousing concepts and dimensional modeling (star schema, slowly changing dimensions)
- * Programming languages: Python or Scala for data pipeline development
- * Data quality frameworks and tools (Great Expectations, Talend, or similar)
- * Version control (Git) and CI/CD practices
- * 5+ years of professional experience as a Data Engineer, with 2+ years building data lakes or enterprise data platforms
- * Demonstrated experience with large-scale data infrastructure projects (100GB+ datasets)
- * Experience implementing data governance, metadata management, and data catalogs
- * Track record of designing systems with high availability (99.9%+ uptime)
- * Strong understanding of data architecture patterns, normalization, and schema design
- * Knowledge of data security, encryption, and compliance frameworks
- * Understanding of database performance tuning and query optimization
- * Excellent written and verbal communication skills
- * Ability to explain complex technical concepts to non-technical stakeholders
- * Strong problem-solving and debugging skills
- * Proactive approach to identifying and resolving data issues
- * Ability to work independently and in cross-functional teams
- * AWS Certified Data Analytics OR Azure Data Engineer or equivalent certification (preferred)
- * Experience in transportation, logistics, or fleet management industries
- * Familiarity with telematics and IoT data processing
- * Experience with data lake solutions (Azure Data Lake Gen2, Microsoft Fabric)
- * Knowledge of streaming data technologies (Kafka, Kinesis, Pub/Sub)
- * Experience with Tableau, Power BI, or other analytics platforms
- * Master's degree in Computer Science, Data Science, or related field
- * Open-source data project contributions or personal data projects
- * Experience with Claude Code or similar AI tools
Benefits
- Comprehensive health insurance
- 401(k) matching
- Professional development budget
- Flexible work arrangements
Company Overview
Company H1B Sponsorship