Note: The job is a remote job and is open to candidates in USA. Kavyos Consulting is seeking a Senior Data Engineer to build and operate an AWS-based data platform and migrate Azure workloads to AWS. The role also develops analytics-ready data models and AI/GenAI data foundations, while delivering secure, governed, cost-efficient infrastructure and pipelines.
Responsibilities
- Help build the AWS data platform: S3 lakehouse on open table formats (Iceberg/Delta) organized in medallion (bronze/silver/gold) layers, with a Redshift, Snowflake, or Databricks serving layer
- Migrate Azure workloads (Data Factory, Synapse, Azure SQL, ADLS) to AWS: assess, re-platform pipelines and models, validate data parity, and support cutover
- Build batch and event-driven pipelines (Glue, Step Functions, Lambda, Kinesis) from Salesforce, SmartSimple and other enterprise sources
- Develop gold-layer star-schema models that support Tableau and self-service analytics
- Build the data foundation for AI: vector pipelines, RAG, and agentic/LLM solutions using Amazon Bedrock, SageMaker, and OpenSearch
- Deliver infrastructure as code and CI/CD (Terraform/CDK, Git) with governance, security, and data-quality controls
- Tune and cost-optimize workloads; document architecture and migration mappings; report progress to stakeholders
Skills
- Deep hands-on AWS data stack: S3, Glue, Redshift, Athena, Lambda, Step Functions, Lake Formation, IAM
- Snowflake and/or Databricks development (Spark/SQL transformations, warehouse and lakehouse models)
- Data modeling: conceptual, logical, and physical design; dimensional (star/snowflake schema) modeling with conformed dimensions and slowly changing dimensions; normalized and Data Vault patterns; defining grain, keys, and semantic-layer models for BI
- Modern techniques: medallion architecture, ELT, change data capture, incremental loads, dbt, data-quality testing
- Cloud architecture: secure, resilient, cost-efficient design (VPC, identity, encryption, multi-account)
- Applied AI/GenAI: Bedrock or SageMaker, LLM integration, RAG, vector databases
- Strong Python (PySpark) and advanced SQL; Parquet, Iceberg, and Delta formats
- Translates business needs into technical solutions; Agile; strong communication and collaboration
- 5+ years running production data platforms on AWS, including at least one cloud migration (Azure-to-AWS preferred)
- Production Snowflake and/or Databricks experience building medallion-layer lakehouses
- Delivered at least one AI/ML or GenAI solution to production (e.g., RAG app, AI agent, ML pipeline)
- Azure data services (Data Factory, Synapse, Azure SQL, ADLS)
- Handling sensitive data and communicating technical concepts to executives and non-technical staff
- Agent frameworks/MCP a plus
- Salesforce integration and Tableau a plus
Company Overview
Company H1B Sponsorship