AI Agent Trajectory Grading Researcher

About the role

Responsibilities

• Develop rubrics and model-based graders for multi-step agent trajectories
• Run human annotation studies to measure grader agreement and bias
• Build datasets of agent runs with known failure modes
• Write up findings for customers and, where appropriate, for publication

Requirements

• Research background in NLP, evaluation or human-AI interaction
• Strong experimental design and statistics
• Hands-on experience building or evaluating LLM agents
• Clear technical writing

Benefits

• Remote within European time zones
• Publication and conference support
• Equity in an independent evaluation company
• Flexible working hours

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.