AI Truthfulness Evaluation Researcher

About the role

Responsibilities

• Design truthfulness evaluation methodology capturing hallucination, confabulation, and misleading statement failure modes
• Build evaluation datasets spanning factual, inferential, and belief-attribution truthfulness assessment
• Develop automated evaluation pipelines running model assessments at scale with human calibration
• Investigate domain specificity: whether truthfulness generalizes across topics or is narrowly calibrated
• Publish evaluation methodology and open-source infrastructure for community adoption
• Engage with the AI safety and NLP research community on evaluation standard development

Requirements

• 6+ years in NLP research or AI evaluation with LLM evaluation, calibration, or safety research experience
• Deep expertise in language model behavior analysis, hallucination characterization, and benchmark design
• Strong Python and statistical analysis skills with large-scale evaluation infrastructure development
• Research track record with peer-reviewed publications in NLP, AI safety, or LLM evaluation
• Ability to identify the limitations in existing evaluation approaches and propose principled improvements

Benefits

• Hybrid: 3 days SF office, 2 days remote
• Top-of-market compensation for AI safety research expertise
• Comprehensive health benefits
• Significant research conference and publication budget
• Equity participation

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.