Note: The job is a remote job and is open to candidates in USA. Calance is seeking a senior, hands-on Data Engineer/Architect to help guide and implement a Databricks-based data platform for treasury management SaaS analytics. The role focuses on end-to-end architecture, scalable data pipelines, governance, reliability, performance optimization, and collaboration with engineering and product teams.
Responsibilities
- Adopt a solution mindset for the Platform Data team by building a robust, scalable, and resilient data platform that serves every application in the portfolio
- Drive design consistency across multi-functional project teams by evaluating solution architecture options and guiding teams to the right one
- Influence at the concept stage. Engage early in concept discussions and proposed technology solutions so that business objectives shape the architecture rather than working around it
- Raise engineering standards through design reviews, code reviews, and mentorship of engineers across the data community
- Stay hands-on. Dive into code with the team and ship data products end to end — not just diagrams and decks
- Design and build ELT/ETL at scale on Databricks, applying medallion (Bronze/Silver/Gold) architecture with well-defined contracts between layers
- Build production-grade pipelines using Delta Lake, Delta Live Tables, Auto Loader, Structured Streaming, and Databricks Workflows, with checkpointing and schema evolution handled deliberately
- Model data for consumption. Design relational, dimensional, and star-schema models that serve both analytical workloads and low-latency, in-product analytics
- Engineer for cost and performance. Tune cluster configuration, job design, table layout, and compute choices; own Delta table maintenance (OPTIMIZE, Z-ORDER/liquid clustering, VACUUM) and platform usage cost
- Automate everything repeatable — CI/CD for notebooks, jobs, and Databricks Asset Bundles; infrastructure as code; environment promotion; automated testing of data and pipelines
- Own governance in Unity Catalog — catalogs, schemas, access control, lineage, audit logging, and masking or tokenization of sensitive fields
- Build data quality in, not on. Implement expectations, validation, anomaly detection, and observability so issues are caught before consumers see them
- Ensure all work meets the quality, operational, security, and architectural standards the organization depends on, including the controls expected of a financial services environment
- Run what you build. Participate fully in the team's YBIYRI (you-build-it-you-run-it) model, including the on-call pager rotation, and drive the operational improvements that reduce toil
- Partner with solution teams, product managers, and business stakeholders to deliver cloud-based data solutions that meet real needs
- Work in an agile, analytical way every day, and continuously improve the team's operations, processes, methodologies, technology choices, and practices
- Share information across boundaries. Interact with your team, customers, and peers to improve cross-department processes
- Champion transparency and partnership between engineering and our product partners
- Stay ahead of the curve. Track technology trends, experiment with new tools, and participate in internal and external technical communities
Skills
- The client needs a senior, hands-on resource who can provide end-to-end architectural guidance
- The desired candidate is a senior hands-on engineer with architectural and design experience, strong end-to-end thinking, and the ability to collaborate with the team, plan ahead, and provide guidance while still delivering implementation work
- You lead with data architecture judgment first, and you back it with depth in distributed data engineering
- You are comfortable owning a technical domain outright, and equally comfortable pairing with an engineer to debug a Spark job
- Data architecture acumen, first and foremost — you can reason about a data estate holistically, not just pipeline by pipeline
- Deep data engineering experience in complex distributed systems, typically 15+ years in data engineering with multiple years designing and operating large-scale platforms in production
- Strong technical acumen that allows you to be accountable for a technical domain and the decisions made within it
- Strong command of the software development lifecycle and engineering excellence practices — version control, testing, code review, CI/CD, observability — applied to data, not just applications
- Proven ability to manage moderately sized feature builds end to end, from ambiguous problem statement to operating system in production
- Strong communication, coordination, and influencing skills, including the ability to explain technical trade-offs to non-technical partners and to build alignment without formal authority
- A track record of collaboration and elevating the teams around you
- Databricks, expert level — Delta Lake and Delta tables, Delta table maintenance, Unity Catalog, access control, compute configuration, and usage cost management
- Automation of ETL/ELT and data movement using medallion architecture and Databricks-native frameworks. Qlik replicate or any other CDC tools
- Languages and frameworks: advanced SQL and Python (PySpark); PowerShell
- Databases and storage: SQL Server, PostgreSQL, and data lake / lakehouse architectures
- Cloud: Azure PaaS services (for example Data Lake Storage, Data Factory, Key Vault, Functions, Event Hubs) and the networking, identity, and secrets management patterns around them
- Data modeling: relational, star-schema and dimensional modeling, and ER diagramming
- BI and analytics tooling: Omni Analytics, Power BI, Tableau, Looker, or similar
- Orchestration, CI/CD, and infrastructure as code — ADO, Octopus deploy or similar, Databricks Workflows or equivalent, Git-based workflows, and Terraform or comparable tooling
- Experience with streaming and near-real-time analytics serving customer-facing workloads
- Databricks Genie Spaces will be nice to have
- Exposure to Omni analytics and embedded/in-product analytics delivery
- Experience with SSRS and migrating legacy reporting estates to a modern lakehouse
- Background in fintech, treasury management, payments, or another regulated financial domain
- Experience with multi-tenant data isolation and row/column-level security in a SaaS product
Benefits
- Remote work (CST hours)
Company Overview
Company H1B Sponsorship