Note: The job is a remote job and is open to candidates in USA. Lumenalta partners with global enterprises to build large-scale data platforms supporting core business products and operations. The Senior Databricks Data Engineer will ingest and model ERP and logistics data from Oracle E-Business Suite and SLICwave in Databricks, connect it to engineering data, and build governed, orchestrated pipelines with documented security controls and lineage.
Responsibilities
- Ingest cost and supplier data from Oracle E-Business Suite into Databricks, using file-based ingestion from S3 extracts or direct read access, including session-level init parameter handling to work within row-level security
- Ingest SLICwave logistics support analysis data into Databricks, working with its relational backend and file exports
- Implement incremental and batch refresh patterns for both sources, starting from representative extracts where direct access is still pending
- Configure schema evolution and baseline data quality checks in Databricks declarative pipelines
- Model cost, supplier, and logistics data so it links to parts, revisions, and BOM structures from Teamcenter and other engineering sources
- Support the architect in presenting the Oracle integration approach to enterprise security reviewers, including documentation of access patterns and controls
- Orchestrate pipelines with Databricks Workflows and ensure lineage is captured for every pipeline
- Work with logistics and ERP SMEs to understand data structures, business rules, and extraction constraints
Skills
- 6+ years in data engineering, including production experience building ingestion pipelines on Databricks
- Working knowledge of Oracle E-Business Suite data structures, including cost, supplier, item, and BOM tables
- Experience with relational ingestion into Databricks from Oracle, SQL Server, or comparable databases
- Understanding of Oracle E-Business Suite's multi-org security model and how it affects returns
- Experience with Auto Loader and file-based ingestion from S3
- Experience with Databricks declarative pipelines or Delta Live Tables, including schema evolution configuration
- Experience modeling ERP or logistics data so it joins to engineering or product data
- Strong PySpark and SQL, and working knowledge of Delta Lake internals such as managed vs. external tables and predictive optimization
- Experience with Databricks Workflows for orchestration
- Clear written communication, including documenting integration designs for security review
- US citizenship
- Fluent English, both written and spoken
- This position supports a U.S. Government contract that requires all personnel to possess U.S. Citizenship
- Familiarity with PLM data, such as Teamcenter parts, BOMs, and revisions
- Familiarity with logistics support analysis (LSA/LSAR) data, such as GEIA-STD-0007 or MIL-STD-1388-2B structures
- SLICwave experience
- Experience migrating Oracle or EDW workloads to Databricks
- Experience with Lakeflow Connect or other CDC-based ingestion tools
- Prior work in a defense, aerospace, or FedRAMP environment, or with CUI or ITAR-controlled data
- Databricks Data Engineer Professional certification
Benefits
- 100% dedicated to one project at a time so that you can innovate and grow.
- Be a part of a team of talented and friendly senior-level developers.
- Work on projects that allow you to use leading tech.
- Delivery Partner Program qualification, sponsored by Lumenalta.
- Databricks Champion candidates receive mentorship from existing Champions through the nomination process.
- Databricks Champion nominees receive a dedicated Databricks solutions architect as a mentor, access to product teams, confidential roadmap briefings, and an invitation to the internal Tech Summit.
- The exam fee is covered for any Databricks certification you want to take.
- This is a fully remote position.
Company Overview