Senior AI Inference Latency Reliability Engineer

Ulvex Systems

Remote · $160k – $200k / year · Full-time · Posted 1 month ago

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

Kubernetes, Triton-style model servers, AWS GPU fleets and a lot of tracing data: that's the surface you'll be responsible for. Ulvex Systems runs an observability platform, and over the past year our own ML features (anomaly detection, log clustering, root cause suggestions) have grown into a serving fleet handling 30,000 requests per second. p99 latency spikes are now our most common internal incident. You'll own the reliability of that serving layer: SLOs, autoscaling, batching strategy, capacity tests and the on-call playbooks that go with them.

Fully remote.

Responsibilities

Own latency and availability SLOs for every ML inference service we run
Tune batching, autoscaling and GPU bin-packing to cut p99 latency without raising cost
Build load tests and capacity models that run before every model release
Lead incident reviews for inference outages and drive the follow-up work

Requirements

Senior SRE or platform experience running latency-sensitive services on Kubernetes
Hands-on experience serving ML models in production on GPUs
Deep familiarity with AWS networking, autoscaling and observability tooling
Calm, methodical approach to incidents and post-mortems