Profile cover
E

AI Multi-Tenant GPU Scheduler Developer

Evan Ferrari

Full-time · Job · Los_Angeles

About the role

Sharing GPU clusters across dozens of teams sounds simple until a rogue training job claims 80% of capacity and blocks everyone else's experiments for six hours. Multi-tenant GPU scheduling is a fairness problem, a packing problem, and a preemption problem simultaneously — and getting it wrong costs both money and team morale. We serve 200+ enterprise ML teams who share GPU infrastructure. Our scheduling layer assigns jobs to GPUs, enforces per-team quotas, preempts low-priority workloads when high-priority jobs arrive, and checkpoints long-running jobs so preemption doesn't mean losing days of training progress. The current scheduler uses a modified fair-share algorithm that has been patched so many times it is fragile. Your first project is a full redesign using a priority-weighted fair queuing approach with dynamic quota rebalancing based on actual utilization. Measurable target: reduce average queue wait time by 40% without increasing cost.

Responsibilities

  • Redesign the multi-tenant GPU scheduler using priority-weighted fair queuing
  • Implement dynamic quota rebalancing based on real-time team utilization
  • Build preemption and checkpointing logic for long-running training jobs
  • Integrate scheduler with Kubernetes custom resource definitions
  • Create fairness monitoring dashboards visible to platform users
  • Benchmark scheduler performance against current system and academic baselines

Requirements

  • 6+ years in platform or infrastructure engineering with scheduling experience
  • Deep knowledge of Kubernetes scheduling, custom schedulers, and resource management
  • Experience with distributed systems, fairness algorithms, or resource allocation
  • Strong Python skills and comfort with Go for Kubernetes operator work
  • Track record of designing systems that handle adversarial usage patterns

Benefits

  • Fully remote
  • Top-of-market compensation
  • Health, dental, and vision
  • $5,000 annual learning budget
  • Equity with 4-year vesting

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python kubernetes docker aws gcp

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

Devops Engineer, US

RemoteAI company

Full-time

Infrastructure Engineer

RemoteAI company

Full-time

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

AI Engineer

RemoteAI company

Full-time

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python kubernetes docker aws gcp mlflow

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile