Opportunity AI Research work from LinkedIn, X, Reddit and Upwork, in one list. Find me work
aimlroles

Member of Technical Staff, Evals and Post-Training

Profound tryprofound.com Open · verified Nov 21, 2026
On-site New York San Francisco Full-time Senior AI Research

At a glance

Senior AI Research role at Profound. New York · full-time · $230,000–$325,000 base.

SalaryStated by Profound
$230,000 – $325,000
Base salary for this role, in USD per year, as published in the posting.
This role, from ProfoundOur estimate for the market: $189,560–$260,250Median $237,500
The range includes the market median of $237,500 for senior AI research roles in New York.
Worked out from 33 postings that stated pay, median $237,500. Last updated Nov 10, 2026.
RoleAI Research
SenioritySenior
LocationNew York
WorkplaceOn-site
EmploymentFull-time
Base pay$230,000–$325,000
PostedOct 29, 2026 · 3w ago
Also hiring inSan Francisco
Role brief

aimlroles summary, based on the employer's posting.

What you'll do

  • Build evaluation workflows with tasks, scoring rubrics, model judges, automated checks and alerts for regressions
  • Assess multi-step agent runs for tool use, analytical accuracy and the quality of their content
  • Shape grading criteria with marketing specialists and data-labeling partners, then convert real usage and failure patterns into reliable evaluations
  • Use evaluation results and expert feedback to create training rewards, preference examples and training environments
  • Develop tools to investigate model behavior and connect evaluations to CI, launch reviews and model selection

What you bring

  • At least two years building evaluation or measurement systems in areas such as AI, experimentation, ranking or search quality
  • Strong Python and data skills, plus experience delivering production systems
  • Deep knowledge of LLM evaluation, including model-as-judge approaches, benchmark design, agent harnesses and calibration
  • Sound experimental design skills and familiarity with supervised fine-tuning, preference optimization and reinforcement learning

Who this fits

This senior role suits a hands-on builder who is comfortable with rapid iteration and making decisions amid noisy, non-deterministic model results. It is an on-site position in New York City or San Francisco.

From the employer

Profound is the fastest-growing AI marketing platform, used by 15% of the Fortune 500 including Comcast, Estée Lauder, and Walmart, and innovators like Ramp, MongoDB, and Figma. It is where marketing teams do the work: research, briefs, content, campaigns, reporting, all in one place, alongside agents that carry the load with them.

Built on 2B+ real user prompts and growing, Profound’s data shows you what people ask AI and what the models say back. Profound's AI Marketer joins that data with deep context about our customers to find the opportunities worth acting on, and then puts agents to work: automating reporting, drafting content, auditing pages for readability, running competitive research. It measures what works and gets better every day.

As a Member of Technical Staff in our Evals and Post-training team, you’ll build the evaluation and measurement systems that tell us how good AI (including Profound's harness, known as Aim) is at marketing tasks. These evals guide what we ship, and they are the foundation for our post-training initiatives: the benchmarks, graders, and environments you build become the signal our models are trained against.

What you’ll do

  • Build Profound’s eval system: tasks, rubrics, LLM judges, programmatic verifiers, regression alerts, and self-improvement loops where we look at the traces when something is wrong and come up with the fix

  • Define what a great AI Marketer run looks like, and analyze trajectories across multi-step tool use, analytical correctness, and content quality

  • Work with marketing experts and data labeling companies to define tasks, metrics, rubrics, and other grading mechanisms

  • Close the loop from usage: input use cases, cluster failure modes, and turn these into reliable evals

  • Turn eval results and expert feedback into training signal: rewards, preference data, and training environments

  • Build tooling for debugging model and agent behavior

  • Make evals part of how we decide things: wired into CI, launch reviews, and model selection

Who you are

  • 2+ years building evaluation or measurement systems, whether AI evals, experimentation, ranking, or search quality

  • Strong Python and data skills, with a track record of shipping production systems

  • Deep familiarity with LLM evaluation: LLM-as-judge, benchmark design, agent harnesses, calibration, and understanding how to integrate them into professional settings

  • Sound experimental design, and the instinct to separate signal from noise in non-deterministic systems

  • Familiarity with modern post-training methods: SFT, preference optimization, and RL

  • Strong opinions about how models and agents should behave, and keeping up with current research

  • Comfortable operating in a fast-paced startup environment with rapid iteration cycles

Location

This is an on-site role based in our NYC or SF office, designed for builders who thrive on speed, iteration, and meaningful impact.

For this role, the expected base salary range is $230,000 to $325,000. Profound’s total compensation package is designed to be competitive and includes base salary, equity, and a full range of benefits and perks. Final compensation will depend on factors such as your skills, experience, qualifications, and location, and will be determined during the interview process. Our recruiting team will share more details about the full compensation package and benefits as you move through hiring.

Note: All official communication from Profound will come from a @tryprofound.com email address. If you're contacted by anyone using a different domain, please disregard and report it as spam.

Opportunity

AI Research work from LinkedIn, X, Reddit and Upwork, in one list.

Each morning you get everything that fits, strongest first.

AI Research$230k+

Find me work

By the team behind aimlroles.

Find me work
Opportunity

AI Research work from LinkedIn, X, Reddit and Upwork, in one list.

Each morning you get everything that fits, strongest first.

Find me work

By the team behind aimlroles.