Note: The job is a remote job and is open to candidates in USA. Railroad19 develops customized software solutions and provides software development services. The company is seeking a Senior Data Engineer to design and build modern GCP lakehouse architectures, data ingestion pipelines, and reusable data-sharing adapters connecting BigQuery data products with Snowflake and Databricks. The role also involves implementing governance, lineage, real-time data ingestion, and end-to-end data engineering features.
Responsibilities
- Design and implement the UniForm write layer (Delta + Iceberg dual metadata)
- Build GCS → BigQuery ingestion pipelines for structured operational datasets
- Develop and implement in Python and Spark
- Implement Kafka-based CDC patterns for real-time and near-real-time ingestion
- Develop data lineage, dependency tracking, and modular adapter code
- Configure Snowflake Horizon external tables for zero-copy reads
- All data hub tables are to be written once using Delta Lake with Iceberg UniForm enabled… readable by all target consumers without conversion
- Implement and certify Delta Sharing endpoints for Databricks consumers
- Build governed access layer components: RBAC, connector registry entries, tenant-scoped authorization
- Align semantic layer models with LookML and KPI catalog definitions
- Collaborate with cross‑functional teams to deliver end‑to‑end features
- Troubleshoot issues across the full stack and contribute to code quality
Skills
- 6+ years of proven enterprise-level experience in Python & Spark
- Advanced experience in GCP BigQuery
- Strong working knowledge of Apache Iceberg, Delta Lake, Iceberg UniForm
- Experience with Delta Sharing; Kafka / CDC pipelines
- Specific work experience in Snowflake Horizon Catalog; Databricks Unity Catalog
- Solid experience with Data lake architecture & ingestion pipeline design
- Excellent Communication skills and the ability to work cohesively with multiple teams
- Prior delivery in enterprise SaaS, media, or advertising technology
- Active daily use of AI-assisted development tools (Claude Code preferred)
Benefits
- 100% remote position
- Medical insurance
- Dental insurance
- Vision insurance
- Pet insurance
- Paid Time Off
- 401(k)
Company Overview