AI/ML Engineer, Individual Contributor
At a glance
Mid-level Machine Learning Engineering role at Clera. San Francisco · full-time · $120,000–$200,000 base.
aimlroles summary, based on the employer's posting.
What you'll do
- Find agent failures such as missing steps, wrong answers, and recurring issues in production traces
- Use rephrasing, abandonment, and retries as signals that an agent may have failed
- Build ways to assess detection quality, including cases without labeled answers
- Test whether suggested fixes address failures, and withhold recommendations when evidence is weak
- Choose models and methods that keep failure detection costs efficient
What you bring
- You have AI/ML engineering experience and have shipped products used by customers
- You have used LLMs in production for evaluations, LLM-as-judge, embeddings, and model selection
- You have operated AI agents in production and handled their failures
- You can judge whether research results are meaningful, including when ground truth is unavailable
Who this fits
This suits an individual-contributor engineer who regularly reviews production traces and uses them to inform product improvements. The role is on-site in San Francisco, and you must be authorized to work in the United States without visa sponsorship.
From the employer
About the Role
Join an AI/ML team building tools to identify and address failures in production AI agents. This role owns the intelligence behind failure detection, evaluation, and proposed fixes, helping make agents more reliable for real-world use.
What You'll Do
Detect subtle agent failures, including omissions, incorrect responses, and patterns across large volumes of traces.
Turn user behavior such as rephrasing, abandonment, and retries into evidence of potential failures.
Build evaluation methods that measure detection quality, including in situations without ground-truth labels.
Verify that proposed fixes reproduce and resolve failures, and avoid recommending changes when evidence is weak.
Improve the cost efficiency of detection by choosing appropriate models and methods for each task.
Review production traces regularly to identify problems and guide product improvements.
What We're Looking For
Demonstrated AI/ML engineering experience, with 0 to 10 years of experience considered.
Experience shipping products that people use and working as an individual-contributor engineer.
Hands-on experience with LLMs in production, including evaluations, LLM-as-judge, embeddings, and deciding when to use smaller models or no model.
Experience running AI agents in production and handling their failures.
Strong research judgment and the ability to distinguish meaningful improvements from noise, including when direct ground truth is unavailable.
Compensation & Benefits
Base salary range: $120,000 to $200,000 annually. Visa sponsorship is not available. The role requires authorization to work in the United States without visa sponsorship.
Location
On-site in San Francisco, United States.