Profile cover
C

Senior AI Inference Latency Reliability Engineer

Caleb Rodriguez

Full-time · Job · Los_Angeles

About the role

Kubernetes, Triton-style model servers, AWS GPU fleets and a lot of tracing data: that's the surface you'll be responsible for. Ulvex Systems runs an observability platform, and over the past year our own ML features (anomaly detection, log clustering, root cause suggestions) have grown into a serving fleet handling 30,000 requests per second. p99 latency spikes are now our most common internal incident. You'll own the reliability of that serving layer: SLOs, autoscaling, batching strategy, capacity tests and the on-call playbooks that go with them.

Responsibilities

  • Own latency and availability SLOs for every ML inference service we run
  • Tune batching, autoscaling and GPU bin-packing to cut p99 latency without raising cost
  • Build load tests and capacity models that run before every model release
  • Lead incident reviews for inference outages and drive the follow-up work

Requirements

  • Senior SRE or platform experience running latency-sensitive services on Kubernetes
  • Hands-on experience serving ML models in production on GPUs
  • Deep familiarity with AWS networking, autoscaling and observability tooling
  • Calm, methodical approach to incidents and post-mortems

Benefits

  • Remote across US time zones
  • On-call compensated with extra pay and time off
  • Equity plus annual refresh
  • Medical, dental, vision and life insurance
  • $2,500 home office budget

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

kubernetes docker python aws mlflow

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Safety Engineer (Back-End Leaning)

RemoteAI company

Full-time

AI Security Engineer Co-Op Intern, Winter 2027

RemoteAI company

Full-time Intern

AI Platform Engineer Co-Op Intern, Winter 2027

RemoteAI company

Full-time Intern

Principal Engineer Team Lead, Simulation Infrastructure

RemoteAI company

Full-time Principal

Platform Reliability Engineer

RemoteAI company

Full-time

Manufacturing Software Engineer Intern - Winter 2027

RemoteAI company

Full-time Intern

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

kubernetes docker python aws mlflow

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile