Note: The job is a remote job and is open to candidates in USA. Noctua Technology's Data Engineering & AI/ML capability supports data-driven operations through data engineering, database management, and infrastructure architecture. The Senior Software Engineer - Data Engineering will construct and optimize data pipelines, implement storage solutions, manage cloud data infrastructure, lead migrations, and collaborate with customers, partners, and internal engineering teams.
Responsibilities
- Acquire, clean, and preprocess diverse datasets from various sources, establishing best practices and ensuring data quality standards
- Design and build required infrastructure for optimal extraction, transformation and loading of data from various data sources using CSP managed services and SQL technologies, focusing on scalability and cost efficiency
- Develop, maintain, and optimize mission-critical data pipelines to ensure a continuous flow of high-quality data, implementing robust monitoring and alerting
- Architect and develop data migration strategies and schemas to lead complex customer migrations from on-prem to cloud technologies
- Oversee and execute high-volume data migration activities, ensuring data integrity and minimal downtime
- Optimize databases and data warehouses for efficient querying and data storage, implementing advanced partitioning, indexing, and tuning techniques
- Perform exploratory data analysis to uncover patterns, trends, and insights, providing actionable recommendations to technical and business stakeholders
- Create visualizations and reports to communicate findings effectively to stakeholders both internally and externally, driving data-driven decision making
- Collaborate with cross-functional teams, including software engineers, domain experts, and business analysts, to understand requirements and deliver integrated solutions, end-to-end data solutions
- Create and maintain comprehensive documentation for data architectures, code, algorithms, and models. Ensure that the knowledge is shared and accessible within the team
- Act on client feedback constructively to improve services and outcome, serving as a key technical contact for data-related discussions
- Continuously seek ways to enhance the overall customer experience, proactively identifying and addressing complex data challenges
- Stay updated on the latest developments in cloud data services,machine learning, data science, and analytics
- Drive innovation by evaluating, proposing, and implementing cutting-edge techniques and technologies to address client challenges
Skills
- Candidates must be based in CA or DC Metro Area for proximity to project and client teams
- Applicants must be US citizens and eligible to obtain and maintain an active Secret security clearance or above
- Expert-level understanding and experience with SQL and relational database concepts
- Deep understanding of database technologies, data warehouses, and ETL tools (e.g., MySQL, PostgreSQL, Beam, Airflow, and Kafka). Proven track record of designing and implementing scalable data pipelines
- Advanced experience with data analysis tools (eg., Jupyter, Colab, Pandas)
- Extensive experience with data visualization tools (eg., Tableau, Looker, PowerBI, Qlik, and SuperSet)
- Demonstrated experience developing comprehensive data strategies and facilitating data migrations into production systems
- Expertise with cloud platforms (e.g., AWS, Azure, GCP)
- Proficiency in programming languages such as Python, Java, or C++
- Strong software engineering skills with an emphasis on writing clean, modular, and maintainable code
- Experience with version control systems (e.g., Git) and collaborative development workflows
- Exceptional problem-solving and critical-thinking skills
- Effective communication skills and ability to work in a collaborative team environment
- Bachelor's or advanced degree in Computer Science, Data Science, Machine Learning, or a related field
- Experience with other database technologies (eg., NoSQL, Graph)
- Any of the below cloud certifications:
- Google Cloud Professional Cloud Architect
- Google Cloud Professional Database Engineer certification
- Google Cloud Professional Data Engineer
- Experience with additional data processing tools and technologies (e.g., Spark, Hadoop)
- Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes)
Benefits
- Primarily Remote. Candidates must be based in CA or DC Metro Area for proximity to project and client teams.
Company Overview