Profile cover
H

AI Metadata Enrichment Pipeline Developer

Harper Adeyemi

Full-time · Job · New_York

About the role

We have 400 million data assets in our catalog — tables, columns, models, dashboards, pipelines — and about 15% of them have meaningful metadata: descriptions, owners, domain tags, quality scores, lineage. The other 85% are orphaned assets that data teams can't evaluate without opening the underlying data, which defeats the purpose of having a catalog. You'll build the metadata enrichment pipeline that automatically populates descriptions, infers domain tags, assigns likely owners based on access patterns, and computes quality signals for unannotated assets. This involves a mix of NLP for description generation from column names and sample data, classification models for domain tagging, and graph analysis for lineage inference. The data governance team is four engineers. You'll own the enrichment pipeline end-to-end: design, build, schedule with Airflow, and monitor output quality. The KPI is increasing the percentage of meaningfully documented assets from 15% to 60% within 12 months.

Responsibilities

  • Build automated description generation for undocumented tables and columns
  • Develop domain classification models for asset tagging
  • Implement owner inference from access patterns and historical edits
  • Create data quality scoring for unannotated datasets
  • Schedule and monitor enrichment pipelines with Airflow
  • Build quality dashboards tracking documentation coverage over time

Requirements

  • 3+ years in data engineering or analytics engineering
  • Strong Python and SQL skills with pipeline development experience
  • Familiarity with NLP techniques for text generation or classification
  • Experience with Airflow or similar workflow orchestration
  • Understanding of data catalog and governance concepts

Benefits

  • Fully remote with flexible schedule
  • Health, dental, and vision insurance
  • 401(k) with match
  • $2,500 professional development fund
  • Regular performance reviews with transparent salary progression

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

data science & analytics MLOps & AI Infrastructure

Skills

python sql spark airflow aws

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

Devops Engineer, US

RemoteAI company

Full-time

Infrastructure Engineer

RemoteAI company

Full-time

PhD Autonomy Engineer Intern - Computer Vision / Deep Learning Summer 2027

RemoteAI company

Full-time Intern

Backend Engineer, Platform

RemoteAI company

Full-time

Job Type

Full-time

Level

Mid-Level

Language

English

Salary Range

$80k – $100k / year

AI Expertise

data science & analytics MLOps & AI Infrastructure

Skills

python sql spark airflow aws pandas

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile