Note: The job is a remote job and is open to candidates in USA. CGI is an IT and business consulting services firm seeking a Databricks Data Engineer to design, develop, and maintain Azure Databricks data solutions. The role focuses on building scalable pipelines for credit card and financial datasets, supporting platform modernization, implementing data quality and reconciliation controls, and protecting sensitive cardholder information.
Responsibilities
- Design, develop, and maintain Azure Databricks pipelines for credit card data ingestion, transformation, integration, and downstream consumption
- Develop scalable processing solutions using PySpark, Python, Spark SQL, SQL, Delta Lake, and ADLS for high volume card datasets
- Build transformation logic for cardholder, account, transaction, authorization, payment, balance, fee, interest, rewards, and servicing data
- Translate business requirements and source to target mappings into reusable Databricks notebooks, workflows, jobs, and ETL/ELT processes
- Support credit card platform conversion and modernization, including data mapping, historical/incremental loads, transformation, and reconciliation
- Implement automated data quality, balancing, reconciliation, cleansing, and exception handling controls across source and target environments
- Protect sensitive cardholder information, including PAN and PCI sensitive data, through appropriate masking, tokenization, encryption, and data handling practices
- Troubleshoot and optimize Databricks pipelines while collaborating with credit card SMEs, architects, analysts, and application teams
Skills
- . 5+ years of Data Engineering, ETL/Data Warehouse, or Big Data experience, including 3+ years of hands on Azure Databricks development
- . Strong hands on development experience with Databricks, PySpark, Python, Spark SQL, SQL, Delta Lake, and ADLS
- . Experience designing and developing scalable ETL/ELT pipelines for large volume transactional and financial datasets
- . Strong experience with data transformation, source to target mapping, cleansing, reconciliation, data quality, and exception handling
- . Credit card/payments domain experience with knowledge of card accounts, transactions, authorizations, payments, balances, billing, and settlement
- . Experience with Azure DevOps, Git, CI/CD, and Agile/Scrum development practices
- . Strong banking/financial services experience and understanding of financial data controls and reconciliation
- . Experience with TSYS or another major credit card processing platform
- . Experience supporting credit card conversions, migrations, or platform modernization
- . Knowledge of the end to end card lifecycle, including authorization, authentication, clearing, settlement, posting, and reconciliation
- . Experience with Mastercard and/or Visa data, interfaces, reporting, rewards, or loyalty programs
- . Understanding of PCI DSS, PAN protection, tokenization/de tokenization, encryption, and sensitive cardholder data management
- . Experience with advanced Databricks capabilities such as Unity Catalog, Workflows, Auto Loader, and Lakeflow/Delta Live Tables
- . Experience modernizing Informatica PowerCenter or SSIS workloads to Databricks/PySpark
- . Azure and/or Databricks certifications and experience with large scale banking conversion initiatives
Benefits
- Comprehensive insurance options
- Matching contributions through the 401(k) plan and the share purchase plan
- Paid time off for vacation, holidays, and sick time
- Paid parental leave
- Learning opportunities and tuition assistance
- Wellness and Well being programs
- This position can be performed remotely from anywhere in the United States.
Company Overview
Company H1B Sponsorship