Profile cover
B

Junior AI Voice Bot Persona Consistency Evaluator

Brandon Kim

Freelance · Job · Chicago

About the role

A voice bot that sounds friendly in some conversations and terse in others, uses formal language with some users and casual language with others, or breaks character when asked difficult questions creates an inconsistent brand experience. Persona consistency — ensuring that a deployed voice bot maintains its defined personality, tone, and communication style reliably across millions of diverse conversations — is harder to measure than it sounds. We deploy enterprise voice bots for 65 clients across customer service, sales, and healthcare verticals. Persona consistency evaluation is a quality assurance gap: we measure accuracy and task completion but don't systematically evaluate whether the bot sounds like itself. You'll help build the evaluation system. This is a freelance role — 20-25 hours per week — building the persona consistency scoring pipeline. You'll define the measurable dimensions of persona consistency (tone formality, vocabulary level, response length consistency, use of defined persona phrases), implement automated scoring on sampled conversation transcripts, and build the weekly persona consistency reports that voice product managers use to identify drift and regression after bot updates.

Responsibilities

  • Define measurable persona consistency dimensions for automated evaluation across conversation samples
  • Build automated scoring pipeline evaluating tone, vocabulary, and style consistency in transcripts
  • Implement persona drift detection identifying when consistency scores change after bot updates
  • Create weekly persona consistency reports for voice product manager review
  • Develop persona regression test suite for pre-release quality checks
  • Track consistency score trends over time as bots are updated and new conversation types emerge

Requirements

  • 1+ year of Python and SQL development experience
  • Basic familiarity with NLP concepts and text analysis
  • Interest in conversational AI, voice products, or UX quality evaluation
  • Attention to consistency and reliability in evaluation metric computation
  • Reliable async communication for a freelance engagement

Benefits

  • Fully remote freelance — 20-25 hours per week
  • Flexible schedule
  • Competitive hourly rate
  • Mentorship from senior voice AI engineers
  • Exposure to enterprise voice bot platform development

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

Automation & AI Workflows AI Product & Strategy

Skills

python sql pandas openai api statistics

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Technical Writer

RemoteAI company

Part-time

Product Manager, Developer Experience

RemoteAI company

Full-time

Software Engineer, Applications (Scientific Development Solutions) (High Seniority)

RemoteAI company

Full-time Senior

PhD Autonomy Engineer Intern - Computer Vision / Deep Learning Summer 2027

RemoteAI company

Full-time Intern

Drone Platform Integration & Network Support Engineer

RemoteAI company

Full-time

Backend Engineer, Platform

RemoteAI company

Full-time

Job Type

Freelance

Level

Junior

Language

English

Salary Range

$60k – $80k / year

AI Expertise

Automation & AI Workflows AI Product & Strategy

Skills

python sql pandas openai api statistics

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile