Profile cover
J

AI Incident Postmortem Quality Scorer

Joseph Kim

Full-time · Job · Chicago

About the role

We're building the next generation of AI-powered developer tooling, and one gap keeps showing up in customer feedback: postmortems that get written and never actually improve anything. You'll build a model that scores incident postmortems against a quality rubric, catching vague root causes and missing action items before the doc gets archived and forgotten. This plugs directly into our existing developer productivity analytics product.

Responsibilities

  • Build a scoring pipeline that evaluates postmortem quality against a defined rubric
  • Own integration with our existing developer analytics product
  • Partner with engineering leads to validate rubric accuracy
  • Iterate on prompts and scoring logic based on real postmortem data

Requirements

  • 3+ years building LLM-powered internal tooling
  • Experience with prompt engineering and structured evaluation
  • Comfortable working with Airflow-based pipelines
  • Familiarity with engineering incident processes a plus

Benefits

  • Fully remote position
  • Competitive salary + equity
  • Health and dental insurance
  • Generous learning budget

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure Automation & AI Workflows

Skills

python langchain prompt engineering docker airflow

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Software Engineer, Full Stack - UK

RemoteAI company

Full-time

Principal Engineer

RemoteAI company

Full-time Principal

AI Resident UK/EMEA

RemoteAI company

Full-time

Software Engineer, Applications (Scientific Development Solutions) (High Seniority)

RemoteAI company

Full-time Senior

PhD Autonomy Engineer Intern - Computer Vision / Deep Learning Summer 2027

RemoteAI company

Full-time Intern

Job Type

Full-time

Level

Mid-Level

Language

English

Salary Range

$100k – $130k / year

AI Expertise

MLOps & AI Infrastructure Automation & AI Workflows

Skills

python langchain prompt engineering docker airflow

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile