Profile cover
D

AI Spot Instance Arbitrage Developer

David Dubois

Full-time · Job · New_York

About the role

GPU spot instances cost 70% less than on-demand — but they can be reclaimed with 2 minutes notice. For ML training jobs that run for hours, a sudden interruption means lost progress, wasted compute, and frustrated researchers. Our arbitrage system makes spot instances reliable: predicting reclamation risk, checkpointing proactively, and migrating jobs between instance types and availability zones before interruptions hit. You'll build the prediction and migration infrastructure: analyzing historical spot pricing and reclamation patterns across regions, predicting reclamation probability windows, designing checkpoint strategies that minimize both storage cost and recovery time, and building the migration orchestration that moves running jobs without requiring researcher intervention. Spot arbitrage is financial engineering meets distributed systems. Every dollar saved on spot instances drops directly to margin. Every interrupted job without a clean checkpoint costs researcher productivity. You'll optimize the tradeoff that determines whether our GPU business is profitable.

Responsibilities

  • Build spot instance reclamation prediction from historical pricing and availability data
  • Design proactive checkpoint strategies minimizing storage cost and recovery time
  • Develop job migration orchestration across instance types and availability zones
  • Analyze cross-region spot pricing for optimal placement decisions
  • Monitor spot savings metrics and interruption impact across client workloads

Requirements

  • 5+ years in cloud infrastructure with GPU workload experience
  • Deep AWS expertise including spot fleet management and pricing APIs
  • Kubernetes skills for workload migration and orchestration
  • Understanding of ML training checkpoint and recovery mechanisms
  • Python proficiency for infrastructure automation and analytics

Benefits

  • Fully remote — US East Coast hours
  • Senior compensation with equity
  • Direct impact on GPU infrastructure profitability
  • $3,000 annual learning budget

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure

Skills

python aws kubernetes docker spark

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

Devops Engineer, US

RemoteAI company

Full-time

Infrastructure Engineer

RemoteAI company

Full-time

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

Software Engineer, Infrastructure

RemoteAI company

Full-time

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

MLOps & AI Infrastructure

Skills

python aws kubernetes docker spark sql

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile