Note: The job is a remote job and is open to candidates in USA. Cantina is a social platform building AI-powered characters for voice, video, and text interactions. The Research / ML Engineer will develop state-of-the-art speech systems for voice conversion and related tasks, owning model development, data strategy, evaluation, production pipelines, and safety guardrails.
Responsibilities
- Model Building: Architect, implement, pre-train, fine-tune, and post-train/alignment (e.g., GRPO/DPO) for large-scale speech models
- Experimental Design: Design, run, and analyze scientific experiments to advance our understanding of the models
- Tool Development: Develop and improve dev tooling to enhance team productivity
- Full-Stack Contribution: Contribute to the entire stack, from low-level optimizations to high-level model design
- Data Ownership: Define data requirements and collaborate on acquisition, curation, augmentation, labeling quality, and synthetic data strategies
- Rigorous Evaluation: Design automated objective/subjective evaluations—listening tests, SV/WER/ASR-based metrics, robustness & bias checks, and red-team studies
- Pipeline Delivery: Harden the training → evaluation → inference pipeline; profile latency, memory, and cost; and meet production SLAs with robust monitoring and rollback
- Safety & Responsibility: Contribute to safety/consent guardrails and to misuse/abuse mitigation for responsible speech technology
Skills
- Exceptional research/development experience with large-scale audio models (>8B parameters, >500k hours of data)
- Deep hands-on experience with diffusion and/or flow-matching transformers, including practical knowledge of samplers, schedules, conditioning mechanisms, and distillation
- Deep hands-on experience training audio VAEs, neural audio codecs, and vocoders latent/tokenizer design, reconstruction and perceptual objectives, adversarial training
- Strong experience with multi-node, multi-GPU distributed training (FSDP/DeepSpeed or equivalent)
- Strong software engineering skills with a proven track record of building complex systems
- Strong with PyTorch and performance work (profiling, CUDA/Triton/C++ as needed) and writing reliable production-quality code
- Shipped large-scale speech/audio or multimodal generative models to production
- Background in working with large-scale ML data, and the ability to iterate on data and triangulate quality using both subjective and objective signals
- Experience with voice cloning, speech control/steerability, or expressive speech generation
- Notable publications and/or open-source contributions in speech/audio/ML
Benefits
- Generous company equity
- Medical, dental, and vision insurance – 99.99% of premiums covered by Cantina
- 42 days of paid time off, including 15 PTO days, 10 sick days, 15 company holidays, and 2 floating holidays
- Generous parental leave & fertility support
- 401(k) retirement savings plan
- Lifestyle spending account – $500/month to use however you’d like
- Complimentary lunch and snacks for in-office employees
- One Medical membership
Company Overview
Company H1B Sponsorship