Profile cover
H

AI API Gateway Performance Engineer

Uelvex Cloud

Remote · $100k – $130k / year · Full-time · Posted 5 months ago

No longer accepting applications

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

Every AI model our clients deploy goes through our API gateway — 890 million requests per month across 200+ models. The gateway handles authentication, rate limiting, request validation, model routing, response caching, and usage metering. When it's slow, every model behind it is slow. When it's down, everything is down. We're looking for an engineer obsessed with latency. Our p50 is 12ms overhead; our p99 is 180ms. That p99 is unacceptable for real-time applications, and the tail latency comes from three sources: cold cache misses, garbage collection pauses, and slow model routing decisions. You'll profile, diagnose, and fix these bottlenecks while adding new gateway features that clients are asking for: request transformation, response streaming, and multi-model fan-out. The platform team is five engineers. You'll own gateway performance as a discipline — setting SLOs, building performance regression tests, and raising the alarm when a code change threatens latency guarantees.

Fully remote.

Responsibilities

  • Profile and eliminate tail latency in the AI API gateway
  • Build performance regression testing into the CI/CD pipeline
  • Implement request transformation and response streaming features
  • Design caching strategies that reduce cold-miss latency spikes
  • Set and monitor latency SLOs across all gateway endpoints
  • Optimize garbage collection and memory management patterns

Requirements

  • 3+ years in backend performance engineering or systems programming
  • Experience profiling and optimizing high-throughput API services
  • Strong understanding of caching, connection pooling, and concurrency
  • Proficiency with Docker, Kubernetes, and cloud infrastructure
  • Comfort with performance measurement tools and methodology

Benefits

  • Fully remote with async-first culture
  • Health, dental, and vision coverage
  • $3,000 annual equipment budget
  • Stock options
  • Flexible PTO policy

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure Automation & AI Workflows

Skills

python docker kubernetes aws fastapi

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

AI Security Engineer Intern (Summer 2027)

Brainco

Full-time Intern

Software Engineer, Infrastructure

Niantic Spatial

Full-time

Senior Infrastructure Engineer

Harvey

Full-time Senior

Senior Infrastructure Engineer

Harvey

Full-time Senior

Research Engineer / Scientist, Pre-training

Anthropic

Full-time

Principal Engineer, Data Platform

AlphaSense

Full-time Principal

Job Type

Full-time

Level

Mid-Level

Language

English

Salary Range

$100k – $130k / year

AI Expertise

MLOps & AI Infrastructure Automation & AI Workflows

Skills

python docker kubernetes aws fastapi mlflow

Ready to apply for this role?

Create a free talent account.

  • Find AI roles in one place
  • Get AI-matched job recommendations
  • Reply to companies that message you
  • Build your public AI talent profile
Create free account & apply Log in

This role is closed. Create your free profile to apply to similar open roles.

Don't miss the next one: get matched to new roles like this.

Create free profile