Profile cover
Z

AI Regression Detection Engineer

Zoe Levy

Full-time · Job · Berlin

About the role

Model performance degrades silently. A retrained model scores well on validation but produces subtly worse predictions in production because the data distribution shifted, a feature pipeline broke, or a labeling standard changed. By the time someone notices, the damage is done. Our regression detection platform catches these problems before deployment. You'll build the automated testing infrastructure: statistical comparison between model versions across stratified evaluation slices, data drift detection between training and serving distributions, and the alerting system that blocks deployments when quality regressions are detected. This is defensive ML engineering — the unglamorous work that prevents expensive production failures. Every model deployment at our clients passes through your regression gates. When the system works, nothing bad happens. When it catches a regression, it saves weeks of debugging and customer trust.

Responsibilities

  • Build automated regression testing for model version comparisons
  • Design stratified evaluation across demographic and behavioral slices
  • Implement data drift detection between training and serving distributions
  • Create deployment gates that block models with detected quality regressions
  • Define regression severity thresholds calibrated to business impact

Requirements

  • 5+ years in ML engineering with model evaluation expertise
  • Strong statistical skills for distribution comparison and hypothesis testing
  • MLflow and Airflow experience for pipeline orchestration
  • Deep understanding of data drift, concept drift, and model degradation
  • Docker and infrastructure skills for testing automation

Benefits

  • Fully remote — European time zones
  • 30 days PTO
  • Senior compensation with equity
  • Preventing production ML failures at scale

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure data science & analytics

Skills

python scikit-learn mlflow docker airflow

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Technical Writer

RemoteAI company

Part-time

AI Resident UK/EMEA

RemoteAI company

Full-time

Software Engineer, Applications (Scientific Development Solutions) (High Seniority)

RemoteAI company

Full-time Senior

Drone Platform Integration & Network Support Engineer

RemoteAI company

Full-time

PhD Autonomy Engineer Intern - Computer Vision / Deep Learning Summer 2027

RemoteAI company

Full-time Intern

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$130k – $160k / year

AI Expertise

MLOps & AI Infrastructure data science & analytics

Skills

python scikit-learn mlflow docker airflow sql statistics

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile