Profile cover
A

AI Document Summarization Quality Engineer

Adam Li

Full-time · Job · Berlin

About the role

PyTorch, LangChain, and structured evaluation pipelines are the core of what we build. If you've ever shipped a summarization feature and then watched it quietly drift into fabricating details six weeks later, this role is about preventing exactly that. You'll build the quality scoring layer that catches unsupported claims, missing key facts, and tone drift across our document summarization product, then feed those signals back into prompt and retrieval tuning before customers ever notice a regression.

Responsibilities

  • Build automated faithfulness scoring for generated summaries
  • Detect unsupported claims and omitted key facts before release
  • Partner with prompt engineers on regression-driven tuning cycles
  • Maintain a growing evaluation dataset across document types

Requirements

  • Experience building llm evaluation or quality-scoring pipelines
  • Comfortable with prompt engineering and openai api integration
  • Understanding of hallucination and faithfulness metrics in nlp
  • Able to work independently across distributed time zones

Benefits

  • Fully remote position
  • Async-first culture with flexible hours
  • Health insurance stipend for EU and US-based hires
  • Annual team offsite

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

NLP & Prompt Engineering

Skills

python langchain hugging face prompt engineering openai api

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Software Engineer, Full Stack - UK

RemoteAI company

Full-time

Team Lead, Actors

RemoteAI company

Full-time Lead

Software Engineer, Applications (Scientific Development Solutions) (High Seniority)

RemoteAI company

Full-time Senior

Senior Autonomy Engineer, Data Curation

RemoteAI company

Full-time Senior

Drone Platform Integration & Network Support Engineer

RemoteAI company

Full-time

PhD Autonomy Engineer Intern - Computer Vision / Deep Learning Summer 2027

RemoteAI company

Full-time Intern

Job Type

Full-time

Level

Mid-Level

Language

English

Salary Range

$100k – $130k / year

AI Expertise

NLP & Prompt Engineering

Skills

python langchain hugging face prompt engineering openai api

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile