Profile cover
E

AI GPU Fleet Manager

Evan Ferrari

Full-time · Job · Los_Angeles

About the role

Managing 400 GPUs across three cloud regions sounds straightforward until a training job in us-east consumes 64 A100s for 72 hours, a client in eu-west needs 16 GPUs for an urgent fine-tuning run, and the spot instance prices in us-west just tripled. Our GPU fleet management system makes allocation decisions every 30 seconds — and every wrong decision costs money or delays a client. You'll own the fleet management layer: real-time GPU allocation across regions, preemption policies that balance priority clients against cost efficiency, spot vs. on-demand arbitrage that minimizes spend without sacrificing availability, and the capacity forecasting system that triggers procurement before demand exceeds supply. This is operations engineering at the intersection of distributed systems and financial optimization. Every GPU-hour has a cost and a revenue opportunity. You'll build the systems that maximize the ratio between the two while keeping every client's SLA green.

Responsibilities

  • Manage real-time GPU allocation across 400+ GPUs in three cloud regions
  • Design preemption and priority policies balancing client SLAs with cost efficiency
  • Build spot vs. on-demand arbitrage systems minimizing infrastructure spend
  • Develop capacity forecasting triggering procurement before demand exceeds supply
  • Monitor fleet utilization metrics and optimize allocation algorithms

Requirements

  • 5+ years in infrastructure engineering with GPU cluster experience
  • Deep Kubernetes expertise for GPU workload orchestration
  • Strong AWS experience including spot instance management
  • Python proficiency for fleet management tooling
  • Understanding of cloud cost optimization strategies

Benefits

  • Fully remote — US West Coast hours
  • Senior compensation with equity
  • $3,000 annual learning budget
  • Managing one of the largest GPU fleets in the industry

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure

Skills

python kubernetes docker aws airflow

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Principal Engineer

RemoteAI company

Full-time Principal

Staff Application Security Engineer

RemoteAI company

Full-time Staff

AI Resident UK/EMEA

RemoteAI company

Full-time

Drone Platform Integration & Network Support Engineer

RemoteAI company

Full-time

Technical Writer

RemoteAI company

Part-time

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$130k – $160k / year

AI Expertise

MLOps & AI Infrastructure

Skills

python kubernetes docker aws airflow sql

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile