Profile cover
D

AI Model Serving Cold Start Latency Engineer

Mivex Cloud

Remote · $160k – $200k / year · Full-time · Posted 2 months ago

No longer accepting applications

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

We just closed our Series B and are scaling fast. We need a senior infrastructure engineer to solve one of our most persistent customer complaints: cold start latency when spinning up new model containers. Every extra second of cold start time shows up directly in customer SLAs, and our current warm-pool approach doesn't scale cost-effectively. You'll redesign how we pre-warm and schedule model containers across multi-cloud GPU capacity, working with our platform team on both the scheduling logic and the cost tradeoffs.

Fully remote.

Responsibilities

  • Redesign container pre-warming strategy to reduce model serving cold start latency
  • Build scheduling logic that balances latency targets against GPU cost
  • Benchmark cold start performance across AWS and GCP GPU instance types
  • Partner with customer-facing teams to align latency targets with SLAs

Requirements

  • 5+ years of experience with Kubernetes-based infrastructure at scale
  • Deep understanding of container scheduling and GPU resource management
  • Experience with multi-cloud deployments across AWS and GCP
  • Track record of reducing latency in production ML serving systems

Benefits

  • Fully remote position
  • Meaningful equity post-Series B
  • Health, dental, and vision coverage
  • Annual infrastructure conference budget
  • 401k matching

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python kubernetes docker aws gcp

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Security Engineer - Detection & Response

Lambda

Full-time Senior

Solutions Engineer, Canada

Glean

Full-time

Research Engineer / Scientist, Pre-training

Anthropic

Full-time

Senior Data Engineer, Data Analytics

Agility Robotics

Full-time Senior

Sr. Solutions Architect - Travel and Hospitality

Databricks

Full-time Senior

Lead/ Senior Machine Learning Scientist, Integrative Phenotyping

insitro

Full-time Lead

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python kubernetes docker aws gcp

Ready to apply for this role?

Create a free talent account.

  • Find AI roles in one place
  • Get AI-matched job recommendations
  • Reply to companies that message you
  • Build your public AI talent profile
Create free account & apply Log in

This role is closed. Create your free profile to apply to similar open roles.

Don't miss the next one: get matched to new roles like this.

Create free profile