Note: The job is a remote job and is open to candidates in USA. Autodesk is a global company known for creating innovative software solutions. They are seeking a Senior Machine Learning QA Engineer to define and uphold quality standards for ML systems, collaborating closely with researchers and engineers to ensure reliable evaluation of ML models and data.
Responsibilities
- Define ML quality strategy and acceptance criteria across data, model, and system levels
- Design and maintain model evaluation suites, metrics, and test datasets
- Evaluating CAD RL model outputs for geometric validity or policy stability
- Defining structured rubrics that translate qualitative findings into measurable evaluation gates
- Testing ML Models from product side
- API Testing
- Automate ML QA workflows using Python and CI/CD (e.g., GitHub Actions, Jenkins)
- Create and maintain test harnesses for ML services and APIs
- Mentor teams on ML QA best practices and consistent evaluation standards
- Build quality gates for training and deployment pipelines (e.g., regression checks, drift detection)
- Contribute to multi-team projects and codebases, ensuring code quality and consistency
- Participate in code reviews and provide constructive feedback to peers
- Document and present findings and ideas across the company
Skills
- Bachelor's degree in Computer Science, Engineering, or equivalent experience
- 7+ years of professional experience in software engineering or QA for ML/AI systems
- Strong programming skills in Python, with experience in test automation
- Familiarity with popular CAD environments tooling
- Proficient in Automation and UAT test suite/framework
- Experience designing QA frameworks or platforms used by multiple teams
- Excellent problem-solving skills and attention to detail
- Strong communication and collaboration skills
- Understanding of software architecture and design patterns
- Ability to work in an agile development environment
- Experience with data validation tooling (e.g., Great Expectations) or labeling workflows
- Familiarity with ML frameworks (e.g., PyTorch, TensorFlow)
- Experience with CI/CD tools and processes
- Experience with data pipelines and orchestration tools (e.g., Airflow, Metaflow)
- Familiarity with MLOps practices (model monitoring, drift, deployment checks)
- Experience with ML evaluation methods, metrics, and benchmarking
- Passion for learning new technologies and improving existing systems
- Experience with cloud providers (e.g., AWS, Azure, Google Cloud Platform)
- Experience testing ML services in production environments
- Knowledge of experiment tracking tools (e.g., Comet, MLflow, Weights & Biases)
Benefits
- Annual cash bonuses
- Commissions for sales roles
- Stock grants
- A comprehensive benefits package
- Autodesk is a hybrid-first company, allowing workers to work remotely, in an office, or a mix of both
Company Overview