Profile cover
E

AI Inference Autoscaling Policy Optimizer

Aevix Cloud

Remote · $160k – $200k / year · Full-time · Posted 5 months ago

No longer accepting applications

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

Static autoscaling rules waste GPU capacity. Scale up too aggressively and you pay for idle GPUs during traffic dips. Scale up too slowly and requests queue during traffic spikes, breaching latency SLAs. The optimal scaling policy depends on traffic patterns that vary by model, time of day, client, and even day of the week — and it needs to anticipate demand, not just react to it. Our predictive autoscaler uses traffic forecasting models to pre-provision capacity before demand arrives. You'll build the forecasting pipeline that predicts request volume per model at 5-minute granularity, the optimization engine that translates forecasts into scaling decisions accounting for GPU warm-up time, and the feedback loop that calibrates predictions against actual traffic. The infrastructure optimization team is five engineers. You'll be the senior ML engineer who brings predictive intelligence to what has traditionally been a reactive systems problem. Every improvement gets measured in two dimensions: cost savings (fewer idle GPUs) and SLA compliance (fewer latency breaches).

Fully remote.

Responsibilities

  • Build traffic forecasting models at 5-minute granularity per model endpoint
  • Design optimization engine translating forecasts into scaling decisions
  • Account for GPU warm-up latency in pre-provisioning timing
  • Implement feedback loops calibrating predictions against actual traffic
  • Measure impact on cost savings and SLA compliance simultaneously
  • Handle special events: client launches, marketing campaigns, seasonal patterns

Requirements

  • 6+ years in ML or systems engineering with autoscaling experience
  • Experience with time series forecasting at high temporal resolution
  • Deep knowledge of Kubernetes autoscaling and GPU resource management
  • Strong Python and optimization skills
  • Understanding of the cost-latency trade-off in model serving

Benefits

  • Fully remote with flexible schedule
  • Top-of-market compensation plus equity
  • Comprehensive health coverage
  • $5,000 annual conference and learning budget
  • Optimize infrastructure handling billions of inference requests

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python kubernetes docker aws pytorch

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Software Engineer

Turing

Full-time

Principal AI Security Engineer

Cerebras

Full-time Principal

Staff Software Engineer

Fivetran

Full-time Staff

Member of Technical Staff, ML Ops

Cohere

Full-time Staff

Member of Technical Staff, GPU Infrastructure

Cohere

Full-time Staff

Kernel Engineer

Cerebras

Full-time

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python kubernetes docker aws pytorch statistics

Ready to apply for this role?

Create a free talent account.

  • Find AI roles in one place
  • Get AI-matched job recommendations
  • Reply to companies that message you
  • Build your public AI talent profile
Create free profile and apply Log in

This role is closed. Create your free profile to apply to similar open roles.

Don't miss the next one: get matched to new roles like this.

Create free profile