Note: The job is a remote job and is open to candidates in USA. NinjaOne unifies IT operations for nearly 40,000 customers across 140+ countries through a platform for endpoint management, patching, backup, and remote access. The Senior Data Engineer will design and scale lakehouse architectures, develop streaming and batch pipelines, manage open table formats, optimize distributed query environments, and improve data platform reliability while collaborating with data and product teams.
Responsibilities
- Lakehouse Architecture: Design and implement Medallion-layer data architectures (Bronze, Silver, Gold), building structured ingestion pipelines and curated data layers that serve downstream analytics and data science workloads
- Pipeline Development: Build and maintain scalable streaming and batch data pipelines using Kafka, Spark, and Airflow, supporting reliable and low-latency data movement across the platform
- Table Format Management: Own and manage open table formats including Iceberg, Delta, and Hudi — selecting the right format for the workload and maintaining performance and reliability at scale
- Query Engine Optimization: Work with Trino/Starburst and Databricks-based environments to optimize large-scale analytics queries and support platform consumers
- Platform Reliability: Monitor pipeline health, troubleshoot data quality and performance issues, and continuously improve the reliability and efficiency of the data platform
- Collaboration: Partner closely with data scientists, analysts, and product teams to understand data requirements and deliver platform solutions that enable data-driven decision-making
- Other duties as needed
Skills
- 10+ years of experience in data engineering with a strong focus on data platform and distributed systems
- Hands-on experience building and managing lakehouse architectures, including Medallion (Bronze/Silver/Gold) layering patterns
- Deep expertise in Apache Kafka, Apache Spark, and Apache Airflow for both streaming and batch pipeline development
- Proficiency in open table formats — Iceberg, Delta Lake, or Hudi — with the ability to evaluate trade-offs and select the right format for the workload
- Experience with Databricks for large-scale data processing and analytics workloads
- Strong proficiency in Python and SQL
- Bachelor's degree in Computer Science, Computer Engineering, Information Technology, or equivalent work experience preferred
- Hands-on experience with Trino or Starburst for distributed query execution
- Familiarity with hybrid streaming/batch architectures and event-driven data systems
- Experience designing Gold-layer datasets optimized for business intelligence and self-serve analytics
Benefits
- Full-time hybrid remote work
- Medical, dental, and vision insurance
- 401(k) plan
- Unlimited PTO
- Opportunity for growth and advancement
- Life insurance coverage
Company Overview