Profile cover
G

Junior AI Data Pipeline Cost Attribution Developer

Grayson Martins

Contract · Job · New_York

About the role

A data platform team running 200 production pipelines often has no idea which pipelines are responsible for 80% of compute costs. Data pipeline cost attribution — mapping cloud resource consumption back to specific pipelines, data products, and the business teams that depend on them — is the foundation of data platform cost governance. Without it, cost optimization is guesswork; with it, engineering and finance can have informed conversations about which pipelines are worth their cost and which should be refactored or deprecated. This is an 8-month contract. We serve 38 enterprise data engineering teams with pipeline management tooling. Cost attribution is a new capability we're adding. We have the cloud billing data and the pipeline execution metadata — the challenge is joining them accurately: a Spark job that ran for 90 minutes might have consumed 240 cluster-minutes of compute across 3 instance types, and attributing that cost to the specific pipeline DAG node that triggered it requires precise execution metadata correlation. As a junior developer, you'll build the cost attribution data pipelines under senior engineer supervision: ingesting cloud billing data, joining it against pipeline execution metadata, and building the cost allocation reports that platform and business teams use to understand their data infrastructure spend.

Responsibilities

  • Build cloud billing data ingestion from AWS Cost Explorer and cloud provider billing APIs
  • Implement pipeline execution metadata correlation joining billing records to specific pipeline runs and DAG nodes
  • Develop compute cost allocation by pipeline, data product, and consuming business team
  • Create cost trend tracking monitoring week-over-week pipeline cost changes and surfacing regression
  • Build cost attribution dashboard for platform team and business stakeholder consumption
  • Maintain attribution pipeline as new pipeline types, cloud services, and cost categories are added

Requirements

  • 1+ year of Python and SQL development experience
  • Basic familiarity with Airflow, cloud billing, or data pipeline concepts
  • Interest in data platform engineering, cost management, or FinOps
  • Attention to cost calculation accuracy and billing data join correctness
  • Reliable async communication for a contract engagement

Benefits

  • Fully remote contract — 8-month engagement
  • Flexible schedule
  • Competitive contract rate
  • Mentorship from senior data platform engineers
  • Potential for full-time conversion

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure data science & analytics

Skills

python sql pandas airflow aws

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Drone Platform Integration & Network Support Engineer

RemoteAI company

Full-time

Technical Writer

RemoteAI company

Part-time

AI Resident UK/EMEA

RemoteAI company

Full-time

Devops Engineer, US

RemoteAI company

Full-time

Software Engineer, Applications (Scientific Development Solutions) (High Seniority)

RemoteAI company

Full-time Senior

Job Type

Contract

Level

Junior

Language

English

Salary Range

$60k – $80k / year

AI Expertise

MLOps & AI Infrastructure data science & analytics

Skills

python sql pandas airflow aws statistics

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile