AI Multi-turn Conversation Coherence Evaluator

About the role

Responsibilities

• Design multi-turn coherence evaluation models analyzing full conversation transcripts for consistency and continuity
• Implement internal consistency detection flagging contradictions across non-adjacent turns
• Build topic threading evaluation measuring how well conversations maintain coherent subject focus
• Develop context tracking accuracy assessment checking whether bots correctly use information from earlier turns
• Create evaluation pipeline processing 4M weekly conversations with coherence quality annotation
• Validate model agreement with human coherence assessors across diverse conversation types and domains

Requirements

• 3+ years in NLP engineering with dialogue system evaluation or conversational AI quality experience
• Strong Python and PyTorch skills with multi-document or multi-turn text modeling experience
• Understanding of dialogue coherence, discourse structure, and conversation quality evaluation
• Experience building evaluation systems that run at production conversation volumes
• Ability to design evaluation frameworks that are both scalable and valid against human judgment

Benefits

• Fully remote across EU time zones
• 30 days PTO plus local holidays
• Private health insurance
• €2,000 annual learning budget
• Equity participation

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.