Profile cover
S

AI Agent Trajectory Grading Researcher

Zenora Labs

Remote · $130k – $160k / year · Full-time · Posted 4 weeks ago

No longer accepting applications

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

We exist to make claims about AI agents measurable. Grading a single answer is a solved problem; grading a 40-step agent run that browsed, called tools and changed its plan twice is not. Our evaluation platform is used by teams deploying agents in customer support, research and coding. Today we mostly score final outcomes, which hides agents that succeed by luck or fail in ways that matter. You'll research and build methods to grade full trajectories: step-level correctness, efficiency, safety violations and recovery from errors. Expect to write internal research notes, run human agreement studies and publish when results warrant it.

Fully remote.

Responsibilities

  • Develop rubrics and model-based graders for multi-step agent trajectories
  • Run human annotation studies to measure grader agreement and bias
  • Build datasets of agent runs with known failure modes
  • Write up findings for customers and, where appropriate, for publication

Requirements

  • Research background in NLP, evaluation or human-AI interaction
  • Strong experimental design and statistics
  • Hands-on experience building or evaluating LLM agents
  • Clear technical writing

Benefits

  • Remote within European time zones
  • Publication and conference support
  • Equity in an independent evaluation company
  • Flexible working hours

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

NLP & Prompt Engineering AI & Machine Learning Engineers

Skills

python openai api langchain statistics pandas

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

AI Researcher - Pre-training (f/m/d)

Aleph Alpha

Full-time

Research Engineer / Scientist, Pre-training

Anthropic

Full-time

Research Engineer, Infrastructure

Harvey

Full-time

Research Engineer / Scientist, Reinforcement Learning

Harvey

Full-time

Research Engineer / Scientist, Reinforcement Learning

Harvey

Full-time

Research Engineer, Infrastructure

Harvey

Full-time

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$130k – $160k / year

AI Expertise

NLP & Prompt Engineering AI & Machine Learning Engineers

Skills

python openai api langchain statistics pandas

Ready to apply for this role?

Create a free talent account.

  • Find AI roles in one place
  • Get AI-matched job recommendations
  • Reply to companies that message you
  • Build your public AI talent profile
Create free account & apply Log in

This role is closed. Create your free profile to apply to similar open roles.

Don't miss the next one: get matched to new roles like this.

Create free profile