AI Bias Auditing Engineer

Arca Works

Remote · $130k – $160k / year · Contract · Posted 5 months ago

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

Every model we deploy goes through a bias audit before it reaches production. That's not aspirational — it's contractual. Our enterprise clients in financial services require documented fairness assessments, and regulators are starting to ask for them too. Right now, our auditing process takes 3 weeks per model. We need to cut that to 3 days without sacrificing rigor. The role is equal parts engineering and methodology. You'll build automated testing pipelines that evaluate models across protected attributes, generate audit reports that satisfy compliance teams, and flag issues before they become PR incidents. You'll also define the standards: which fairness metrics matter for which use case, what thresholds trigger a block, and how to remediate without destroying model performance. This is a 12-month contract with strong potential for conversion. You'll report to our Head of Responsible AI and work alongside two policy analysts and one ML engineer.

Fully remote.

Responsibilities

Build automated bias testing pipelines for pre-deployment model evaluation
Define fairness metrics and thresholds appropriate to each use case
Generate audit reports that meet regulatory and client compliance requirements
Create remediation playbooks for common bias patterns
Train engineering teams on bias-aware development practices
Monitor deployed models for fairness drift over time

Requirements

5+ years in ML engineering with focus on fairness, accountability, or explainability
Deep knowledge of fairness metrics: demographic parity, equalized odds, calibration
Experience building automated testing and evaluation pipelines
Strong statistical background with ability to communicate findings to non-technical audiences
Familiarity with regulatory frameworks like EU AI Act or NYC Local Law 144