Profile cover
E

AI Inference Response Caching Engineer

Evan Ferrari

Full-time · Job · Los_Angeles

About the role

PyTorch, Kubernetes, and a fleet of inference endpoints serving billions of requests a month are the core of what we build. If you've ever obsessed over cache hit rates the way other engineers obsess over model accuracy, this role is for you. You'll design the caching layer that sits in front of our inference services and directly reduce the GPU hours we burn on repeated requests.

Responsibilities

  • Design and ship a response caching layer for high-volume inference endpoints
  • Improve cache hit rates without compromising response freshness
  • Profile GPU savings from caching changes and report impact
  • Own the FastAPI services fronting our largest inference workloads

Requirements

  • 6+ years building high-throughput backend or ML infrastructure
  • Deep understanding of caching strategies and cache invalidation
  • Production experience with PyTorch model serving
  • Comfortable owning services under heavy production load

Benefits

  • Fully remote position
  • Top-of-market salary + equity
  • Health and dental insurance
  • Annual GPU conference budget
  • Sabbatical after 4 years

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure

Skills

python pytorch docker kubernetes fastapi

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

Principal Engineer

RemoteAI company

Full-time Principal

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

Senior AI Researcher- Pre-training (f/m/d)

RemoteAI company

Full-time Senior

Staff Application Security Engineer

RemoteAI company

Full-time Staff

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$200k+ / year

AI Expertise

MLOps & AI Infrastructure

Skills

python pytorch docker kubernetes fastapi

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile