Profile cover
C

LLM Evaluation Engineer - Compliance Reviews

Carl Bauer

Full-time · Mid-level · Paris

No longer accepting applications

This role is no longer accepting applications

Similar open roles you can apply to now:

See all open AI jobs

About the role

We already have the models. What we need now is someone obsessive enough to measure them properly. This role is about building evaluation harnesses, test sets, and failure analysis for compliance-oriented LLM workflows where a smooth demo means very little if the edge cases are weak.

Responsibilities

  • Build repeatable evaluation workflows for compliance review assistants
  • Create benchmark sets from real historical cases and reviewer feedback
  • Analyse regressions across prompts, model versions, and retrieval changes
  • Partner with product and compliance teams on acceptance criteria
  • Raise the overall discipline of how LLM quality is discussed internally

Requirements

  • 3+ years in AI engineering, evaluation, or NLP tooling
  • Python for evaluation harnesses and internal tools
  • OpenAI API experience in production settings
  • Prompt Engineering for structured and auditable outputs
  • RAG understanding for evidence-grounded responses
  • SQL for storing and comparing evaluation outcomes

Benefits

  • $116,000 - $138,000 base salary
  • Remote-first in Europe
  • Annual learning budget
  • Private health insurance
  • A narrow, important problem space with strong quality standards

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

NLP & Prompt Engineering

Skills

python openai api prompt engineering rag sql

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

FinOps Data Engineer

RemoteAI company

Full-time

Principal Engineer Team Lead, Simulation Infrastructure

RemoteAI company

Full-time Principal

Senior AI Researcher: LLM post-training for hallucination reduction (f/m/d)

RemoteAI company

Full-time Senior

Manufacturing Software Engineer Intern - Winter 2027

RemoteAI company

Full-time Intern

Software Engineer Intern - Summer 2027

RemoteAI company

Full-time Intern

Director, Product Management - Data Transformation (Frontier Engineering)

RemoteAI company

Full-time Executive

Job Type

Full-time

Level

Mid-level

Language

English

Salary Range

$116,000 - $138,000

AI Expertise

NLP & Prompt Engineering

Skills

python openai api prompt engineering rag sql

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile