Note: The job is a remote job and is open to candidates in USA. The hiring client is a confidential IT services and consulting company. The Senior Data Engineer will architect and implement scalable data platforms using AWS, Databricks, and Palantir Foundry, aligning secure and cost-efficient solutions with business outcomes.
Responsibilities
- Architect role in a global enterprise focusing on design and implementation of scalable data platforms using AWS, Databricks and Palantir Foundry in a hybrid work model for day shift with no travel requirement, ensuring secure, reliable and cost efficient solutions that align with business outcomes and social impact objectives
- Design robust data architectures on AWS that support analytics, reporting and machine learning workloads while ensuring scalability, performance and security compliance
- Develop end to end data processing solutions using Databricks that optimize data ingestion, transformation and aggregation for diverse business domains
- Coordinate integration of Palantir Foundry platforms with existing enterprise systems to enable unified data views and reusable analytical assets
- Implement data modelling practices that standardize reference data, master data and domain specific schemas to support consistent decision making across teams
- Optimize cloud resource usage on AWS by refining storage patterns, compute configurations and scheduling strategies to balance cost and performance
- Create reusable templates and patterns for data pipelines using Databricks notebooks and jobs to accelerate project delivery and promote engineering standards
- Establish monitoring practices and observability dashboards for data pipelines and platforms so that data quality, latency and availability expectations are continuously met
- Guide application and data teams in adopting architectural principles such as modular design, resilience and security by design to reduce operational risks
- Collaborate with stakeholder groups to translate business requirements into technical blueprints that clearly define data flows, service responsibilities and integration points
- Document architecture decisions, diagrams and data lineage in an accessible form so that engineering and governance teams can maintain and evolve the platforms effectively
- Review code, configurations and infrastructure definitions for AWS, Databricks and Palantir Foundry implementations to ensure compliance with enterprise standards
- Align architectural roadmaps with company sustainability and social impact goals by supporting data products that improve transparency, efficiency and ethical insights
- Participate in hybrid collaboration rituals using digital tools and on site sessions to maintain effective communication, knowledge sharing and continuous improvement
Skills
- Experience: 6 to 10 Yrs
- Possess professional experience of six to ten years working in data engineering or data architecture roles with emphasis on large scale cloud environments
- Demonstrate strong practical expertise in core AWS services related to compute, storage, networking and security used for modern data platforms
- Exhibit hands on proficiency with Databricks including cluster configuration, notebook development, job orchestration and performance tuning for batch and streaming workloads
- Show proven experience implementing data solutions with Palantir Foundry such as data onboarding, ontology design and operational analytic workflows
- Apply solid understanding of data warehousing, lakehouse patterns, data governance and metadata management to design consistent enterprise data ecosystems
- Use programming skills in languages such as Python and SQL to build, test and troubleshoot complex data transformations and integrations across platforms
- Display familiarity with modern DevOps and data operations practices including version control, continuous integration and automated testing for data pipelines
Benefits
- Health Benefits
- Referral Program
- Excellent growth and advancement opportunities
- Hybrid work model
Company Overview
Company H1B Sponsorship