Profile cover
S

AI Evaluation Engineer

Zenora Labs

Hybrid · $60k – $80k / year · Contract · Posted 5 months ago

No longer accepting applications

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

How do you know if an AI system is actually good? That's the question we help companies answer. We build evaluation frameworks for LLM-powered products — testing for accuracy, hallucination rates, bias, latency, and cost. Our clients include mid-size SaaS companies who've shipped AI features and now need to know if they're working. This is an entry-level role for someone who's analytical, detail-oriented, and curious about AI quality. You'll start by running evaluation pipelines, analyzing results, and writing reports. Over time, you'll design new evaluation metrics and build automated testing harnesses. We're a team of 5, and this is a contract-to-hire position. If you're someone who finds bugs that others miss and enjoys the satisfaction of quantifying quality, you'll do well here.

Responsibilities

  • Run LLM evaluation pipelines and analyze results for client reports
  • Track and report on accuracy, hallucination rates, and bias metrics
  • Build automated test harnesses for AI feature validation
  • Maintain evaluation datasets and golden test sets
  • Present findings to clients with clear visualizations

Requirements

  • Bachelor's degree in CS, math, statistics, or related field
  • Python proficiency with pandas for data analysis
  • Basic understanding of LLMs and prompt engineering
  • Analytical mindset with strong attention to detail
  • Good written communication for client-facing reports

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

AI & Machine Learning Engineers Automation & AI Workflows

Skills

python openai api langchain sql pandas

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Sr Technical Program Manager, Data

Hyperscience

Full-time Senior

Principal Engineer Team Lead, Behavior Safety & Metrics

Motional

Full-time Principal

Materials Lead

Lambda

Full-time Lead

Lead/ Senior Machine Learning Scientist, Integrative Phenotyping

insitro

Full-time Lead

AWS Alliances Lead

LangChain

Full-time Lead

AWS Alliances Lead

LangChain

Full-time Lead

Job Type

Contract

Level

Junior

Language

English

Salary Range

$60k – $80k / year

AI Expertise

AI & Machine Learning Engineers Automation & AI Workflows

Skills

python openai api langchain sql pandas statistics

Ready to apply for this role?

Create a free talent account.

  • Find AI roles in one place
  • Get AI-matched job recommendations
  • Reply to companies that message you
  • Build your public AI talent profile
Create free account & apply Log in

This role is closed. Create your free profile to apply to similar open roles.

Don't miss the next one: get matched to new roles like this.

Create free profile