AI GPU Memory Optimization Engineer

Mivex Cloud

Remote · $130k – $160k / year · Full-time · Posted 4 months ago

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

A 70B parameter model that fits in theory doesn't always fit in practice. GPU memory fragmentation, activation checkpointing decisions, optimizer state placement, mixed-precision strategy — each choice interacts with the others in ways that are hard to predict without deep hardware knowledge. Our clients lose 20-40% of their training throughput to suboptimal memory configuration, and most of them don't know it. We run training infrastructure for 55 enterprise ML teams across A100 and H100 clusters. GPU memory optimization is the highest-leverage work on our platform: the same cluster that trains a model in 9 days can train it in 6 days with the right memory strategy. You'll build the profiling and optimization tooling that closes this gap. Your work: instrument training runs to build detailed memory utilization profiles, identify fragmentation patterns and inefficient allocation sequences, recommend and implement optimization strategies per workload, and build the automated configuration tuning system that applies memory best practices without requiring ML engineers to become hardware experts. The feedback loop is tight — you can see the clock speed difference within hours.

Fully remote.

Responsibilities

Build GPU memory profiling tooling capturing fragmentation, allocation, and utilization patterns during training
Identify and implement memory optimization strategies per model architecture and batch configuration
Design automated configuration tuning that applies memory best practices across new workloads
Evaluate trade-offs between gradient checkpointing, mixed precision, and optimizer state partitioning
Create memory utilization benchmarks comparing configurations across A100 and H100 hardware
Advise client ML teams on memory-efficient training architecture decisions

Requirements

6+ years in ML systems, GPU computing, or distributed training infrastructure
Deep understanding of PyTorch memory management, gradient checkpointing, and optimizer state
Experience with CUDA memory profiling tools and GPU hardware architecture
Familiarity with DeepSpeed, FSDP, or similar distributed training frameworks
Track record improving training efficiency through hardware-aware optimization