Research Engineer, Benchmarks
At a glance
Mid-level AI Research role at Clera. Singapore · full-time · $100,000–$170,000 base.
aimlroles summary, based on the employer's posting.
What you'll do
- Build and maintain benchmarks that reveal how AI agents perform on specialized, realistic workflows
- Turn input from subject-matter experts into practical tasks and standards for judging agent performance
- Create dependable infrastructure to run models and agents through benchmark tasks at scale
- Analyze benchmark difficulty, reliability, and failure patterns, then check findings against real-world evaluations
- Explain benchmark methods and outcomes in clear technical reports and documentation
What you bring
- Two to four years in software engineering, machine learning engineering, or research, with two years building benchmarks or agent environments
- Practical experience designing and running evaluations for AI agents or large language models
- Strong Python, Docker, and Linux skills for building and operating reliable infrastructure
- Experience creating evaluation metrics, statistical analyses, or validation studies
Who this fits
This role suits someone who works independently and can draw out domain workflows by collaborating with subject-matter experts. You’ll work on-site in Singapore, and visa sponsorship is available.
From the employer
About the Role
Join a small, technical team working on AI evaluation and reinforcement learning infrastructure. You will design and build rigorous benchmarks for assessing AI agents on realistic, domain-specific workflows, helping research and engineering teams understand how well agents perform in practice.
What You'll Do
Design, implement, and maintain benchmarks for evaluating AI agents on domain-specific tasks.
Work with subject-matter experts to translate real workflows into practical tasks and evaluation criteria.
Build reliable infrastructure to run models and agents against benchmark tasks at scale.
Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.
Validate benchmark results against real-world evaluations and user needs.
Write clear technical documentation and reports that explain methods and results.
What We're Looking For
Two to four years of experience in software engineering, machine learning engineering, or research, including at least two years building AI benchmarks, evaluation infrastructure, or agent environments.
Hands-on experience designing and running evaluations or benchmark environments for AI agents or large language models.
Strong proficiency with Python, Docker, and Linux, including building and operating reliable infrastructure.
Experience developing metrics, statistical analyses, or validation studies for evaluation systems.
Experience collaborating with subject-matter experts and understanding workflows across technical or business domains.
Strong technical writing, careful attention to detail, and the ability to work independently in an early-stage, fast-moving environment.
Research publications or technical writing on benchmarking, model evaluation, or failure modes are beneficial. Experience with reinforcement learning pipelines or public benchmark projects is also useful.
Compensation & Benefits
Base salary: $100,000 to $170,000 USD annually. Visa sponsorship is available.
Location
On-site in Singapore, Singapore.