Profile cover
A

AI Serverless Cold Start Prediction Engineer

Aiden Ferrari

Full-time · Job · New_York

About the role

Serverless ML inference saves money when traffic is bursty — you only pay for what you use. But cold starts kill the user experience: the first request after an idle period takes 8-15 seconds while the container spins up, the model loads, and the GPU warms up. Our prediction system anticipates when cold starts will happen and pre-warms containers before traffic arrives. You'll build the traffic prediction models that forecast when each serverless endpoint will receive its next request, the pre-warming logic that balances cost savings against cold start probability, and the feedback system that learns from prediction errors. The optimal strategy depends on the endpoint's traffic pattern: high-volume endpoints benefit from keep-alive policies, while low-volume endpoints benefit from predictive pre-warming timed to their usage cycles. The serverless platform team is four engineers. You'll own the prediction and pre-warming intelligence while others handle the container lifecycle management and the billing system.

Responsibilities

  • Build traffic prediction models forecasting next-request timing per endpoint
  • Design pre-warming strategies balancing cost savings against cold start risk
  • Classify endpoint traffic patterns for strategy selection
  • Implement feedback loops learning from cold start occurrences
  • Measure impact on cold start rate and infrastructure cost simultaneously
  • Handle special patterns: scheduled traffic, webhook-driven, user-initiated

Requirements

  • 5+ years in backend or infrastructure engineering with ML experience
  • Understanding of serverless architectures and cold start dynamics
  • Experience with time series prediction or arrival process modeling
  • Strong Python and cloud infrastructure skills
  • Ability to optimize for competing objectives: cost vs latency

Benefits

  • Fully remote with flexible hours
  • Health and dental insurance
  • $2,500 professional development budget
  • Stock options
  • Solve a problem every serverless ML team faces

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python aws docker kubernetes sql

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

Devops Engineer, US

RemoteAI company

Full-time

Software Engineer, Infrastructure

RemoteAI company

Full-time

Senior AI Researcher- Pre-training (f/m/d)

RemoteAI company

Full-time Senior

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$80k – $100k / year

AI Expertise

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python aws docker kubernetes sql statistics

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile