Note: The job is a remote job and is open to candidates in USA. CGI is a global IT and business consulting services firm that delivers technology and business solutions. The Databricks Data Engineer will design, develop, and maintain Azure Databricks pipelines for high-volume credit card data, support platform conversions and modernization, and implement data quality, reconciliation, security, and exception-handling controls.
Responsibilities
- Design, develop, and maintain Azure Databricks pipelines for credit card data ingestion, transformation, integration, and downstream consumption
- Develop scalable processing solutions using PySpark, Python, Spark SQL, SQL, Delta Lake, and ADLS for high volume card datasets
- Build transformation logic for cardholder, account, transaction, authorization, payment, balance, fee, interest, rewards, and servicing data
- Translate business requirements and source to target mappings into reusable Databricks notebooks, workflows, jobs, and ETL/ELT processes
- Support credit card platform conversion and modernization, including data mapping, historical/incremental loads, transformation, and reconciliation
- Implement automated data quality, balancing, reconciliation, cleansing, and exception handling controls across source and target environments
- Protect sensitive cardholder information, including PAN and PCI sensitive data, through appropriate masking, tokenization, encryption, and data handling practices
- Troubleshoot and optimize Databricks pipelines while collaborating with credit card SMEs, architects, analysts, and application teams
Skills
- . 5+ years of Data Engineering, ETL/Data Warehouse, or Big Data experience, including 3+ years of hands on Azure Databricks development
- . Strong hands on development experience with Databricks, PySpark, Python, Spark SQL, SQL, Delta Lake, and ADLS
- . Experience designing and developing scalable ETL/ELT pipelines for large volume transactional and financial datasets
- . Strong experience with data transformation, source to target mapping, cleansing, reconciliation, data quality, and exception handling
- . Credit card/payments domain experience with knowledge of card accounts, transactions, authorizations, payments, balances, billing, and settlement
- . Experience with Azure DevOps, Git, CI/CD, and Agile/Scrum development practices
- . Strong banking/financial services experience and understanding of financial data controls and reconciliation
- . Bachelor's Degree
- . Experience with TSYS or another major credit card processing platform
- . Experience supporting credit card conversions, migrations, or platform modernization
- . Knowledge of the end to end card lifecycle, including authorization, authentication, clearing, settlement, posting, and reconciliation
- . Experience with Mastercard and/or Visa data, interfaces, reporting, rewards, or loyalty programs
- . Understanding of PCI DSS, PAN protection, tokenization/de tokenization, encryption, and sensitive cardholder data management
- . Experience with advanced Databricks capabilities such as Unity Catalog, Workflows, Auto Loader, and Lakeflow/Delta Live Tables
- . Experience modernizing Informatica PowerCenter or SSIS workloads to Databricks/PySpark
- . Azure and/or Databricks certifications and experience with large scale banking conversion initiatives
Benefits
- Comprehensive insurance options
- Matching contributions through the 401(k) plan and the share purchase plan
- Paid time off for vacation, holidays, and sick time
- Paid parental leave
- Learning opportunities and tuition assistance
- Wellness and Well being programs
- This position is open to candidates located anywhere within the United States and may be performed remotely.
Company Overview