AI Active Learning Sampling Engineer

Jaltrix Data

Remote · $100k – $130k / year · Full-time · Posted 4 months ago

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

Labeling data is expensive. Our clients spend $0.04-$0.12 per labeled example and produce 50,000 new labels per week, which means annotation budgets are meaningful. The question of which examples to label next — from a pool of millions of unlabeled candidates — determines whether you need 100,000 labels or 40,000 to reach the same model accuracy. Active learning is the answer, and most of our clients aren't using it. You'll build the active learning sampling infrastructure that sits between our clients' unlabeled data pools and their annotation queues. The system needs to support multiple query strategies — uncertainty sampling, core-set selection, diversity-aware sampling — and integrate with our annotation workflow platform to deliver the right candidates to the right annotator pools at the right priority. The technical challenge is scale: evaluating 10 million candidate examples per day across multiple strategies requires efficient batching, approximate inference, and smart caching. The business challenge is proving ROI: you'll build the evaluation framework that shows clients exactly how many labels they saved versus random sampling.

Fully remote.

Responsibilities

Build an active learning sampling system supporting multiple query strategies
Implement uncertainty, diversity, and core-set sampling at 10M candidate scale
Integrate sampling pipeline with annotation workflow APIs
Build approximate inference for efficient candidate scoring across large unlabeled pools
Create ROI evaluation framework comparing label efficiency vs. random sampling baseline
Monitor sample quality and adapt strategies as models improve during training

Requirements

3+ years in ML engineering with experience in active learning or data-efficient training
Strong Python and PyTorch skills
Experience building ML inference at scale with batching and optimization
Understanding of active learning literature and query strategy trade-offs
Ability to design systems that integrate with existing annotation workflow platforms