Machine Learning Engineer, Core Evaluations
At a glance
Mid-level Machine Learning Engineering role at Cantina. Europe · full-time.
Pay not stated Machine Learning Engineer salary
aimlroles summary, based on the employer's posting.
What you'll do
- Build evaluation pipelines for speech generation and recognition models in both development and production
- Create user studies to assess subjective model quality and turn product needs into measurable metrics
- Develop dashboards that automate evaluation reporting and let teams compare model results
- Train models to assess additional metrics, then use findings to guide model and data teams
- Coordinate with product managers on measurement and help establish the evaluation team
What you bring
- You have strong judgment in choosing metrics that reflect model performance
- You have designed user studies through Mechanical Turk or a comparable service
- You can train and fine-tune models, with statistical expertise to compare results and guide decisions
- You bring strong programming and engineering skills, plus experience training ASR or TTS models
Who this fits
This founding evaluation role suits an engineer ready to shape evaluation practices and help grow a team that may later be led by you. Experience on large-scale machine learning projects is also listed as a qualification.
From the employer
About Cantina:
Cantina Labs is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism. We bring characters to life, transforming how people tell stories, connect, and create. We build and power ecosystems. Cantina, our flagship social AI platform, is just the beginning.
If you're excited about the potential AI has to shape human creativity and social interactions, join us in building the future!
About the Role:
We are seeking an experienced Machine Learning Engineer (MLE) to focus on audio model evaluation, specifically for speech generation and recognition models.
This role involves designing and developing comprehensive model evaluation pipelines for both development and production environments, as well as creating automated dashboards for reporting evaluation results.
As the founding member of our evaluation team, the ideal candidate is expected to leverage their experience to lead our evaluation efforts and play a key role in the future growth of the evaluation team.
What You’ll Do:
Designing model evaluation pipelines for models in development and production
Designing user studies for subjective model evaluations.
Converting requirements into measurable metrics.
Designing and developing automated evaluation dashboard to see model performances and compare results.
Training new models to capture new and different evaluation metrics.
Communicating with the model team to help design better models based on the evaluation results.
Communicating with the data team to help decide the type of data necessary to improve model performance.
Communication with the product-manager to make sure product requirements are correctly measured.
Help grow the evaluation team as the founding member.
Lead the evaluation team in the future.
What You’ll Bring:
Strong experience and intuition for designing metrics that capture model performance.
Strong experience with designing user studies on Mechanical Turk or similar platforms. .
Strong experience with model training and fine-tuning for model evaluation.
Strong statistical knowledge and experience to statistically compare evaluation results and take decisions.
Very strong engineering and programming skills.
Experience with training ASR, TTS models.
Experience at ML teams working on large-scale machine learning problems. (>3B models with >1m hours of data)
Compensation:
When determining compensation, a number of factors will be considered, including skills, experience, job scope, location, and competitive compensation market data.
Benefits for U.S.-based roles:
Competitive salary and generous company equity
Medical, dental, and vision insurance – 99.99% of premiums covered by Cantina
42 days of paid time off, including:
15 PTO days
10 sick days
15 company holidays
2 floating holidays
Generous parental leave & fertility support
401(k) retirement savings plan
Lifestyle spending account – $500/month to use however you’d like
Complimentary lunch and snacks for in-office employees
One Medical membership, and more!