Profile cover
Z

AI Model Performance Profiling Tool Developer

Zoe Levy

Full-time · Job · New_York

About the role

ML engineers know their model is slow but can't tell you why. Is it the attention layers? The tokenizer? The data loading? The GPU memory thrashing? Our profiling tool instruments the full inference path — from API request to GPU kernel execution — and produces flamegraph-style visualizations that pinpoint exactly where time is spent. The tool needs to work across model types (transformers, CNNs, GBMs), serving frameworks (Triton, vLLM, TorchServe), and deployment environments (Kubernetes, bare metal, serverless). You'll build the instrumentation layer that hooks into these different stacks, the aggregation pipeline that collects profiling data without significantly impacting inference latency, and the visualization frontend that makes bottlenecks obvious. The developer tools team is four engineers. You'll own the profiling backend — instrumentation, data collection, and analysis. The frontend developer handles visualization; the platform engineers handle integration with our deployment system. Every improvement gets measured by how fast users can identify their inference bottleneck.

Responsibilities

  • Build instrumentation for profiling across model types and serving frameworks
  • Design low-overhead data collection that doesn't impact inference latency
  • Create analysis pipelines that identify bottlenecks from profiling data
  • Support multiple deployment environments: Kubernetes, bare metal, serverless
  • Develop profiling APIs used by the visualization frontend
  • Test profiling accuracy against known synthetic bottlenecks

Requirements

  • 3+ years in systems engineering with profiling or observability experience
  • Understanding of GPU execution, CUDA, and model inference pipelines
  • Experience with PyTorch internals or model serving frameworks
  • Strong Python skills with performance measurement methodology
  • Familiarity with Kubernetes and container-based deployments

Benefits

  • Fully remote with async communication
  • Health, dental, and vision insurance
  • $3,000 annual learning budget
  • Stock options
  • Build tools used by ML engineers daily

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python pytorch docker kubernetes aws

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

Senior AI Researcher- Pre-training (f/m/d)

RemoteAI company

Full-time Senior

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

Devops Engineer, US

RemoteAI company

Full-time

AI Engineer

RemoteAI company

Full-time

Job Type

Full-time

Level

Mid-Level

Language

English

Salary Range

$100k – $130k / year

AI Expertise

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python pytorch docker kubernetes aws mlflow

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile