Note: The job is a remote job and is open to candidates in USA. VideoAmp is the tech-first measurement company transforming how advertising is valued, bought, and sold. The Principal Database Infrastructure Engineer will serve as a technical cornerstone of VideoAmp's Database Infrastructure team, driving the design and execution of scalable, production-critical data systems that power VideoAmp's platform.
Responsibilities
- Design and implement the physical plan distribution pass, including network shuffle, coalesce, and partition isolator insertion. Own the plan serialization codec that ships sub-plans to workers, and maintain the S3 and Flight result exchange paths
- Own worker discovery and heartbeating, and lead development of the next generation of load balancing: a work-stealing protocol and a replication-aware hash ring, both currently in design, to keep workers evenly loaded and resilient to node loss
- Work across cost-based join reordering, cross-stage bloom filter cascade, scan deduplication, and selectivity estimation. Several of these live in our DataFusion fork; you will upstream where it makes sense and maintain the delta where it does not
- Own the NVMe LRU cache over S3, Parquet read strategies including full-file and range reads, and Iceberg partition pruning and snapshot handling
- Close the remaining gap on queries where we still trail Snowflake, specifically multi-shuffle plans and redistribution after scalar-subquery extraction. TPC-H benchmarks are the scorecard
Skills
- 8+ years of software engineering experience with significant depth in database infrastructure, distributed systems, or data platform engineering
- Strong systems programming in Rust, or deep C++ or Go experience with a clear path to Rust, including async runtimes such as Tokio and concurrent data structures
- Proven experience building or significantly modifying a distributed data system such as a query engine, stream processor, distributed database, or large-scale data pipeline, with a solid understanding of shuffles, partitioning, and network and memory bottlenecks
- Fluency with columnar formats and vectorized execution, including Arrow, Parquet, and the mechanics behind their performance characteristics
- Strong grounding in distributed systems fundamentals: consistent hashing, leader and heartbeat protocols, backpressure, partial failure, and graceful degradation
- A performance engineering mindset: you profile before optimizing and defend changes with real benchmark numbers
- Direct experience with Apache DataFusion or another SQL query planner or optimizer such as Spark Catalyst, Calcite, Trino, ClickHouse, or DuckDB
- Experience with Apache Iceberg or a comparable open table format such as Delta Lake or Hudi
- Familiarity with Kubernetes and cloud infrastructure including EKS, S3, and IRSA, particularly on ARM or Graviton
- Query optimizer experience including join ordering, predicate pushdown, and cardinality or selectivity estimation
Benefits
- Equity participation included
- Discretionary & flexible PTO + Spring, Summer & Winter company breaks
- Inclusive and comprehensive medical, dental & vision
- 401(k) with matching
- HSA & FSA
- Paid Maternity & Parental Leave for all family additions
- Cell phone & wifi reimbursement
- Commuter benefits
Company Overview