AI Test Failure Root Cause Classifier

Joltura Labs

Remote · $130k – $160k / year · Full-time · Posted 4 months ago

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

A test suite that fails 200 times per day is useful only if engineers can quickly determine which failures are real bugs versus flaky infrastructure versus environment noise. Today most teams triage manually: read the failure message, check recent commits, decide whether to investigate. At scale — thousands of test runs per day across a distributed CI system — manual triage is impossible. Our test intelligence platform processes test failure data for 340 engineering organizations. The root cause classifier we are building needs to categorize failures into: real regressions introduced by the change under test, pre-existing flakiness in the test itself, infrastructure issues like network timeouts or database connection failures, and environment-specific failures tied to test parallelism or resource contention. You will own the classifier architecture: designing features from test failure messages, stack traces, and contextual metadata; building the multi-class model; and developing the feedback loop from engineer triage decisions that continuously improves accuracy.

Fully remote.

Responsibilities

Design multi-class test failure classifier from failure messages and stack traces
Build contextual features using commit metadata and historical failure patterns
Implement flaky test detection model as a specialized failure category
Create engineer feedback loop that converts triage decisions to training signal
Design evaluation framework measuring classification accuracy on confirmed root causes
Integrate classifier into CI/CD dashboard with actionable per-failure explanations

Requirements

5+ years in ML engineering with NLP or log analysis system experience
Experience with text classification on noisy, domain-specific technical text
Strong Python and PyTorch skills
Deep understanding of software testing, CI/CD systems, and common failure patterns
Track record shipping ML classifiers in developer tool or platform contexts