Profile cover
C

AI Changelog Summarization Quality Engineer

Charlotte Tran

Full-time · Job · UTC

About the role

We don't care where you work from. We care about whether the changelog summaries our system generates are actually accurate. We already auto-generate changelogs from commit history and PR descriptions for hundreds of client repos. What's missing is a quality layer that catches when the summary misrepresents what actually shipped. Your role is to build the evaluation and correction pipeline that flags low-confidence summaries for review before they reach a client's public release notes.

Responsibilities

  • Build quality scoring for auto-generated changelog summaries
  • Design a review queue for low-confidence summaries
  • Fine-tune prompting strategies to reduce summary inaccuracies
  • Analyze failure patterns across different repo types
  • Work async with the docs team on review workflows

Requirements

  • Experience with LLM-based summarization systems
  • Familiarity with prompt engineering and OpenAI API
  • Comfortable working fully async
  • Some experience with developer tooling or dev-facing products

Benefits

  • Fully remote
  • Flexible hours across any time zone
  • Health coverage
  • Annual team retreat

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

NLP & Prompt Engineering Automation & AI Workflows

Skills

python langchain openai api prompt engineering sql

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Software Engineer, Full Stack - UK

RemoteAI company

Full-time

Technical Writer

RemoteAI company

Part-time

Software Engineer, Applications (Scientific Development Solutions) (High Seniority)

RemoteAI company

Full-time Senior

Drone Platform Integration & Network Support Engineer

RemoteAI company

Full-time

PhD Autonomy Engineer Intern - Computer Vision / Deep Learning Summer 2027

RemoteAI company

Full-time Intern

Backend Engineer, Platform

RemoteAI company

Full-time

Job Type

Full-time

Level

Junior

Language

English

Salary Range

$60k – $80k / year

AI Expertise

NLP & Prompt Engineering Automation & AI Workflows

Skills

python langchain openai api prompt engineering sql

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile