Profile cover
R

Freelance LLM Evaluation Specialist - Claims Triage

Ryan Ashford

Freelance · Mid-level · Chicago

No longer accepting applications

This role is no longer accepting applications

Similar open roles you can apply to now:

See all open AI jobs

About the role

An insurance workflow team needs short-term help improving how it evaluates LLM-assisted claim triage suggestions. The system is live, but the team lacks strong evaluation structure and needs someone who can bring discipline quickly. This project suits someone who knows how to define good test cases, compare model behaviour, and write clear recommendations.

Key Deliverables

List the expected deliverables for this project.

  • Audit the current evaluation approach for claims triage suggestions
  • Create a realistic test set with severity, ambiguity, and edge-case coverage
  • Compare prompt and workflow variants with clear scoring criteria
  • Recommend changes to grounding, escalation, and output formatting
  • Deliver a concise handoff plan the internal team can continue using

Requirements

Technical stack needed for this mission.

  • Hands-on experience evaluating LLM or NLP workflows in production settings
  • Prompt Engineering and failure analysis skills
  • OpenAI API familiarity
  • Python for evaluation scripts and dataset preparation
  • SQL for pulling examples and annotating outcomes
  • Ability to turn ambiguous quality concerns into practical test plans

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

NLP & Prompt Engineering

Skills

prompt engineering openai api python sql rag

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

AI Software Engineer

RemoteAI company

Full-time

Applied AI Architect (Brazil)

RemoteAI company

Full-time

Senior/Staff Data Scientist

RemoteAI company

Full-time Staff

Technical Program Manager

RemoteAI company

Full-time

AI Associate, Office of the CEO

RemoteAI company

Full-time

Forward Deployed Engineer, Data De-Identification

RemoteAI company

Full-time

Contract Type

Fixed-price

Level

Mid-level

Budget Range

$12,000 - $18,000

Duration

6-8 weeks

AI Expertise

NLP & Prompt Engineering

Skills

prompt engineering openai api python sql rag

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile