AI Model Performance Comparison Dashboard Developer

Gravex Cloud

Remote · $100k – $130k / year · Full-time · Posted 4 months ago

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

An ML team running 40 experiments needs to answer one question: which model should we deploy? Answering it requires comparing performance across held-out sets, production traffic samples, latency benchmarks, and cost profiles simultaneously — and doing it in a UI that doesn't require an engineering degree to interpret. Most experiment tracking tools show you the numbers. They don't help you make the decision. Our ML platform serves 55 enterprise teams, and the model comparison interface is consistently the feature users cite when they switch from Weights & Biases or MLflow. We have a working comparison view. It's not good enough: it handles metric comparison well but falls apart on model behavior comparison — where do the two models disagree, on which input types, and what does that pattern suggest about which is better for the deployment use case? You'll own the comparison dashboard: building behavioral disagreement analysis that surfaces where two models differ on real inputs, implementing sliced evaluation views that compare performance across demographic and feature segments, and designing the recommendation interface that synthesizes comparison signals into a deployment recommendation with explicit reasoning.

Fully remote.

Responsibilities

Build behavioral disagreement analysis surfacing where candidate models diverge on real production inputs
Implement sliced evaluation views comparing model performance across user-defined feature segments
Design recommendation interface synthesizing comparison signals into reasoned deployment suggestions
Create cost-vs-performance trade-off visualization for models with different latency and accuracy profiles
Build comparison sharing and annotation features for collaborative model review
Develop comparison dashboard APIs consumed by CI/CD deployment gates

Requirements

3+ years in ML platform engineering, data visualization, or developer tooling
Strong Python and TypeScript skills with dashboard or analytics product development experience
Understanding of ML evaluation methodology and experiment tracking concepts
Experience designing interfaces that help non-expert users make complex technical decisions
Familiarity with MLflow, Weights & Biases, or similar experiment tracking tools