Note: The job is a remote job and is open to candidates in USA. VAILEXA is seeking a Data Engineer, GenAI Data Systems professional to support hands-on data and software engineering initiatives. The role focuses on building and operating data pipelines, models, databases, dashboards, telemetry, annotation workflows, and automation solutions across large-scale multimodal datasets, while collaborating with internal and external stakeholders.
Responsibilities
- Build and maintain data pipelines and processing frameworks supporting dataset creation, annotation workflows, and large scale data movement across batch and incremental processing patterns
- Design and maintain data models and pipelines for large scale multimodal datasets including text, image, video, and audio
- Design and support high volume logging and telemetry systems with a focus on operational efficiency, monitoring, and troubleshooting
- Develop and maintain dashboards, reports, metrics, and alerting solutions that track throughput, quality, cost, and overall program health
- Design and maintain data models, schemas, databases, lakehouse environments, dimensional models, and reporting structures
- Build, optimize, and support Databricks workloads including transformations, orchestration, governance, and performance tuning
- Implement data quality validation, monitoring, and alerting mechanisms to improve reliability and trust in data assets
- Develop and maintain workflows that move data into and out of annotation platforms including preprocessing, post processing, validation, and delivery of annotated datasets
- Support engineering needs related to annotation project user interfaces, testing activities, and quality assurance initiatives
- Develop backend services, automation tools, and production software using Python and established engineering best practices
- Troubleshoot complex technical issues spanning multiple systems and identify root causes to implement sustainable solutions
- Partner with business, engineering, and operational teams to gather requirements, communicate progress, and document systems and procedures
- Independently evaluate ambiguous requests, design solutions, manage priorities, and execute cross functional initiatives
- Integrate pipelines and applications with cloud platforms, storage systems, and data annotation technologies while ensuring strong observability and reliability
Skills
- • Software + data engineering experience: Ideally, someone who has held both types of roles. The immediate need is hands-on operational data engineering, but we'd also like someone who can contribute to horizontal tooling development
- • Large-company experience: Someone comfortable navigating complex environments, working across internal and external teams, and using a mix of internal and third-party tools. Experience at a company of client's scale is a plus
- • Hands-on engineering focus: We want someone who can hit the ground running with tactical development and operational execution. Please prioritize candidates who have recently been building and maintaining data systems rather than focusing primarily on architecture, technical leadership, or AI/ML strategy
- • Experience level: Although the JD specifies 4+ years, I'd generally target ~2–8 years of relevant experience, with 4–6 years being the likely sweet spot. Someone with 8+ years should not necessarily be ruled out, but they should be very comfortable with hands-on, tactical engineering work
- • Most recent role: Ideally, the candidate's most recent role is focused on data engineering or software engineering involving data tooling/systems
- • Bachelor's Degree in Computer Science, Engineering, or a related field, or equivalent professional experience
- • Four or more years of professional experience in Data Engineering or a related technical discipline
- • Proven experience designing, building, and operating production ETL and ELT pipelines supporting large and complex datasets
- • Strong proficiency in SQL and Python
- • Experience with version control systems, testing frameworks, and continuous integration and continuous deployment practices
- • Experience with workflow orchestration tools such as Airflow, Dagster, Databricks Jobs, or similar platforms
- • Experience designing and maintaining data models, databases, warehouses, or lakehouse architectures
- • Experience supporting high volume logging or telemetry solutions
- • Experience building dashboards, reports, metrics, and alerting solutions
- • Hands on experience with Databricks, Spark, Delta Lake, or similar large scale data platforms
- • Experience working with AWS, Azure, or Google Cloud Platform
- • Demonstrated proficiency using AI tools for coding and data related workflows while applying strong engineering fundamentals and technical judgment
- • Strong problem solving and troubleshooting skills
- • Excellent written and verbal communication skills
- • Ability to work independently, manage competing priorities, and proactively communicate status and risks
- • Experience supporting human in the loop annotation programs and data labeling workflows
- • Direct experience with SuperAnnotate
- • Experience supporting machine learning, Large Language Model, or Vision Language Model training data pipelines
- • Experience working with large scale multimodal datasets including text, image, video, and audio
- • Experience with JavaScript, TypeScript, HTML, or full stack application development supporting internal tools and workflow applications
- • Technical degree: Bachelor's degree in Computer Science or a closely related field preferred
- • Large-company experience: Someone comfortable navigating complex environments, working across internal and external teams, and using a mix of internal and third-party tools. Experience at a company of client's scale is a plus
- • Eight or more years of Data Engineering or Analytics Engineering experience
Benefits
- Remote work arrangement; occasional in-person collaboration with team members is optional.
Company Overview
Company H1B Sponsorship