Profile cover
E

AI Inference Cold-Start Latency Engineer

Evan Ferrari

Full-time · Job · Berlin

About the role

Kubernetes, PyTorch and a custom model loader written in-house are the backbone of Aevix Cloud. We run inference for customers who scale from zero to thousands of requests in seconds, and right now the first request on a fresh replica can take 40 seconds. That is the problem you will own. You'll profile container start-up, weight loading, CUDA context creation and graph compilation, then build the fixes: snapshot-based restores, lazy weight streaming, smarter pre-warming. It is deep systems work with very visible results, and our customers notice every second you cut.

Responsibilities

  • Profile and break down cold-start time across container, runtime and model layers
  • Build weight streaming and snapshot restore paths for our largest model families
  • Drive adoption of the new loader across all inference clusters with clear rollout metrics
  • Write internal postmortems and design docs that the rest of the infra team can build on

Requirements

  • Solid experience running GPU workloads on Kubernetes in production
  • Comfortable reading PyTorch internals and debugging CUDA initialization issues
  • Experience with container image optimization and layered filesystems
  • Python required; familiarity with Go or Rust is welcome

Benefits

  • Fully remote within UTC-1 to UTC+3
  • Equity plus annual cash bonus
  • Home office stipend of €1,500
  • 30 days paid holiday
  • Conference travel budget

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure

Skills

python kubernetes docker pytorch gcp

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior AI/ML Architect, Applied Field Engineering

RemoteAI company

Full-time Senior

Manager, Forward Deployed Engineering

RemoteAI company

Full-time

Software Engineer 1 - Cloud Infrastructure

RemoteAI company

Full-time

Platform Engineer

RemoteAI company

Full-time

Member of Technical Staff - Production Engineering

RemoteAI company

Full-time Staff

Senior Software Enginer, Data and Integrations

RemoteAI company

Full-time Senior

Job Type

Full-time

Level

Mid-Level

Language

English

Salary Range

$130k – $160k / year

AI Expertise

MLOps & AI Infrastructure

Skills

python kubernetes docker pytorch gcp fastapi

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile