AI Inference Autoscaling Policy Optimizer

About the role

Responsibilities

• Build traffic forecasting models at 5-minute granularity per model endpoint
• Design optimization engine translating forecasts into scaling decisions
• Account for GPU warm-up latency in pre-provisioning timing
• Implement feedback loops calibrating predictions against actual traffic
• Measure impact on cost savings and SLA compliance simultaneously
• Handle special events: client launches, marketing campaigns, seasonal patterns

Requirements

• 6+ years in ML or systems engineering with autoscaling experience
• Experience with time series forecasting at high temporal resolution
• Deep knowledge of Kubernetes autoscaling and GPU resource management
• Strong Python and optimization skills
• Understanding of the cost-latency trade-off in model serving

Benefits

• Fully remote with flexible schedule
• Top-of-market compensation plus equity
• Comprehensive health coverage
• $5,000 annual conference and learning budget
• Optimize infrastructure handling billions of inference requests

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.