Profile cover
S

AI Tabular Data Synthesis Benchmark Engineer

Savannah Watanabe

Contract · Job · New_York

About the role

Everyone claims their synthetic data generator is the best, but there's no standard benchmark for comparing them. Utility metrics (does a model trained on synthetic data perform as well as one trained on real data?) conflict with privacy metrics (can an attacker re-identify individuals from the synthetic output?), and different generators excel on different data types. We're building the definitive benchmark. You'll design the evaluation framework: selecting reference datasets that cover the diversity of real-world tabular data (sparse, imbalanced, high-dimensional, multi-table), defining utility and privacy metrics with clear mathematical specifications, implementing evaluation pipelines that produce reproducible results, and creating the leaderboard that our clients use to select the right generator for their use case. This is an 8-month contract. You'll work with our research team who develop the generators and our product team who need clear, defensible quality claims for enterprise sales.

Responsibilities

  • Design comprehensive benchmark datasets covering diverse tabular data characteristics
  • Define utility and privacy metrics with rigorous mathematical specifications
  • Build reproducible evaluation pipelines for generator comparison
  • Create leaderboard infrastructure for public and private benchmarking
  • Analyze generator performance across data types and use cases
  • Document benchmark methodology for academic and commercial credibility

Requirements

  • 3+ years in data science with evaluation methodology experience
  • Strong statistical background for metric design and analysis
  • Experience with synthetic data, privacy, or generative model evaluation
  • Proficiency in Python and pandas for benchmark implementation
  • Ability to write clear methodology documentation

Benefits

  • Fully remote contract position
  • Competitive monthly rate
  • Contribute to industry-standard benchmarking
  • Work with cutting-edge synthetic data research
  • Potential for publication of benchmark results

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

data science & analytics MLOps & AI Infrastructure

Skills

python statistics pandas scikit-learn docker

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

PhD Autonomy Engineer Intern - Computer Vision / Deep Learning Summer 2027

RemoteAI company

Full-time Intern

Backend Engineer, Platform

RemoteAI company

Full-time

Product Manager, Developer Experience

RemoteAI company

Full-time

Data Scientist/Data Engineer (Mid to Sr Levels)

RemoteAI company

Full-time Senior

Job Type

Contract

Level

Mid-Level

Language

English

Salary Range

$80k – $100k / year

AI Expertise

data science & analytics MLOps & AI Infrastructure

Skills

python statistics pandas scikit-learn docker sql

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile