Profile cover
A

AI Rollback Automation Engineer

Aurora Vestergaard

Full-time · Job · New_York

About the role

Model deployments fail in ways that aren't obvious. A model ships to 10% of traffic, looks fine on standard metrics, but two hours later a specific user segment is getting worse predictions. By the time someone notices and escalates, the full rollout has happened. A manual rollback takes 40 minutes, and every minute of bad predictions erodes user trust. Our rollback automation system reduces that window to under 3 minutes — automated anomaly detection, decision logic for triggering rollback, rollback execution, and a post-incident report generated before the incident bridge even ends. We serve 65 platform engineering teams at enterprise tech companies. You'll own the detection-to-rollback pipeline: improving the statistical models that distinguish real regressions from natural variance, building the decision rules that balance false positive risk against rollback latency, and designing the rollback execution path that works across Kubernetes, SageMaker, and Vertex AI deployment targets.

Responsibilities

  • Improve statistical models for distinguishing real regressions from metric variance
  • Build decision logic for automated rollback trigger conditions
  • Design rollback execution paths for Kubernetes, SageMaker, and Vertex AI
  • Implement automated post-incident reports generated on rollback completion
  • Create shadow-mode testing for rollback system changes before production
  • Monitor rollback system reliability and false positive/negative rates

Requirements

  • 3+ years in MLOps or platform engineering with ML deployment experience
  • Experience with Kubernetes, Docker, and cloud ML deployment services
  • Strong Python skills and statistical reasoning for anomaly detection
  • Understanding of ML serving infrastructure and deployment patterns
  • Systematic debugging mindset for complex distributed system failures

Benefits

  • Fully remote
  • Health and dental from day one
  • $3,000 annual learning budget
  • Home office stipend
  • Quarterly bonus tied to system reliability metrics

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure Automation & AI Workflows

Skills

python kubernetes docker aws mlflow

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

Devops Engineer, US

RemoteAI company

Full-time

Infrastructure Engineer

RemoteAI company

Full-time

Drone Platform Integration & Network Support Engineer

RemoteAI company

Full-time

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

Job Type

Full-time

Level

Mid-Level

Language

English

Salary Range

$80k – $100k / year

AI Expertise

MLOps & AI Infrastructure Automation & AI Workflows

Skills

python kubernetes docker aws mlflow fastapi

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile