Note: The job is a remote job and is open to candidates in USA. Openmind Technologies Inc. is seeking a Principal Data Software Engineer to lead the design and development of core data platform capabilities. The role focuses on building reusable Python tools, SDKs, libraries, ingestion frameworks, data quality tooling, connectors, resilient pipelines, and internal platform components while driving architecture and mentoring engineers.
Responsibilities
- Architect, design, and build internal Python tools, SDKs, and libraries to be consumed across the organization by other engineering teams
- Create metadata-driven ingestion frameworks and reusable ETL/ELT components
- Build data quality toolkits, schema validation libraries, and internal orchestration utilities for platforms like Airflow or Prefect
- Develop custom connectors (API, SFTP, Kafka, Snowflake) and API wrappers for S3, GCS, Kafka, and data warehouses
- Build CLI tools for pipeline debugging and automated testing harnesses for data workflows
- Apply advanced Python software engineering principles (OOP, design patterns, dependency injection) to data platform design
- Implement automated testing strategies, including pytest, mocks, and CI/CD automation pipelines
- Design resilient, production-grade pipelines featuring retries, exponential backoff, idempotency, logging, monitoring, and error handling
- Optimize systems for performance and concurrency using asyncio, multiprocessing, and query execution tuning
- Drive architectural decisions for the data platform while setting standards for modular architecture, packaging, and dependency management
- Translate broad business and analytics needs into scalable data platform features and tooling
- Mentor senior engineers, lead technical design discussions, and own the technical roadmap for internal data tooling
Skills
- Software Engineering Foundations: 10 years+ of Extensive experience as a Python developer building reusable components, internal libraries, and APIs (FastAPI/Flask)
- Data Engineering Stack: Deep experience with SQL (window functions, query plans), distributed compute (Spark/PySpark), orchestration tools (Airflow/Prefect), data transformation frameworks (dbt), and object storage (S3/GCS)
- Distributed Systems & Streaming: Hands-on experience with technologies like Kafka (topics, partitions, consumer groups) and distributed architectures
- Software Development Practices: Proven track record in test-driven development (TDD), CI/CD pipelines, versioning, modular architecture, and configuration management
- Leadership & Influence: Proven ability to influence product and engineering teams, articulate design patterns, and mentor engineering talent
- History of building internal Python libraries and data ingestion frameworks utilized by other software engineers
- Experience designing custom connectors, API wrappers, and data quality/validation tooling
- Deep familiarity with distributed systems, architectural decision-making, and test-driven development in a data environment
- Proven capability to optimize Python code for high performance and system reliability
Company Overview
Company H1B Sponsorship