Profile cover
Z

AI Test Failure Root Cause Classifier

Zoe Levy

Full-time · Job · New_York

About the role

A test suite that fails 200 times per day is useful only if engineers can quickly determine which failures are real bugs versus flaky infrastructure versus environment noise. Today most teams triage manually: read the failure message, check recent commits, decide whether to investigate. At scale — thousands of test runs per day across a distributed CI system — manual triage is impossible. Our test intelligence platform processes test failure data for 340 engineering organizations. The root cause classifier we are building needs to categorize failures into: real regressions introduced by the change under test, pre-existing flakiness in the test itself, infrastructure issues like network timeouts or database connection failures, and environment-specific failures tied to test parallelism or resource contention. You will own the classifier architecture: designing features from test failure messages, stack traces, and contextual metadata; building the multi-class model; and developing the feedback loop from engineer triage decisions that continuously improves accuracy.

Responsibilities

  • Design multi-class test failure classifier from failure messages and stack traces
  • Build contextual features using commit metadata and historical failure patterns
  • Implement flaky test detection model as a specialized failure category
  • Create engineer feedback loop that converts triage decisions to training signal
  • Design evaluation framework measuring classification accuracy on confirmed root causes
  • Integrate classifier into CI/CD dashboard with actionable per-failure explanations

Requirements

  • 5+ years in ML engineering with NLP or log analysis system experience
  • Experience with text classification on noisy, domain-specific technical text
  • Strong Python and PyTorch skills
  • Deep understanding of software testing, CI/CD systems, and common failure patterns
  • Track record shipping ML classifiers in developer tool or platform contexts

Benefits

  • Fully remote
  • Health, dental, and vision
  • $4,000 annual learning and conference budget
  • Equity package
  • Flexible working hours

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

AI & Machine Learning Engineers Automation & AI Workflows

Skills

python pytorch hugging face fastapi docker

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior AI Researcher- Pre-training (f/m/d)

RemoteAI company

Full-time Senior

Technical Writer

RemoteAI company

Part-time

Senior Product Security Engineer

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

Senior Machine Learning Engineer (Clinical Team)

RemoteAI company

Full-time Senior

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$130k – $160k / year

AI Expertise

AI & Machine Learning Engineers Automation & AI Workflows

Skills

python pytorch hugging face fastapi docker sql

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile