Note: The job is a remote job and is open to candidates in USA. Bayside Solutions is recruiting for a Streaming Data Platform Engineer role; the hiring client is not identified. The engineer will build and operate cloud-native, real-time streaming platform capabilities, including reusable services and self-service tools that support internal data and machine learning consumers.
Responsibilities
- Design, build, and operate distributed streaming systems using Apache Flink and/or Apache Spark Structured Streaming for high-throughput, low-latency workloads
- Develop production-grade platform services, APIs, reusable frameworks, and SDKs using Python and/or Java (Spring Boot)
- Contribute to control-plane and data-plane architecture, including workload orchestration, configuration, lifecycle management, and operational reliability
- Enable scalable data management for ML consumers: dataset organization, efficient access and retrieval, indexing patterns, and repeatable training-data workflows
- Engineer observability, fault tolerance, checkpointing, recovery, scaling, performance optimization, and automated testing for distributed processing systems
- Partner with platform teams, application engineers, and ML practitioners to translate use cases into self-service capabilities and well-supported developer interfaces
- Apply security and governance practices, including authentication, authorization, data protection, and appropriate handling of sensitive datasets
Skills
- Strong hands-on real-time streaming experience with Apache Flink or Apache Spark Structured Streaming in production; able to explain design tradeoffs and incidents
- Deep understanding of distributed data processing, partitioning, state management, event-time processing, backpressure, and failure recovery
- Strong software engineering proficiency in Python or Java, including API/service design, automated tests, and maintainable code
- Experience building data platform capabilities, internal developer platforms, reusable libraries/SDKs, or large-scale backend systems - beyond individual pipelines
- Experience with microservices, containers, cloud-native deployment, CI/CD, monitoring, and production operations
- Ability to work across requirements ambiguity, evaluate architecture options, and communicate tradeoffs with engineering stakeholders
- Hands-on knowledge of both Flink and Spark, plus Databricks or comparable managed distributed-data platforms
- Experience with Java/Spring Boot and Python in the same platform ecosystem
- Experience supporting ML practitioners through training-data access, feature/dataset management, data quality, indexing, or governed data interfaces
- Familiarity with Kafka or comparable event-streaming infrastructure; schema evolution and exactly-once/at-least-once delivery tradeoffs
- Experience with Kubernetes, service orchestration, cloud object storage, data access governance, and RBAC/ABAC patterns
- Bayside Solutions, Inc. is not able to sponsor any candidates at this time
- Candidates for this position must qualify as a W2 candidate
Benefits
- Remote role
Company Overview