Profile cover
M

AI Human-in-the-Loop Orchestration Engineer

Mason Oliveira

Full-time · Job · Los_Angeles

About the role

Quelva Works builds workflow automation tools used by enterprise operations teams in insurance, financial services, and healthcare. In these industries, full automation isn't always safe — some decisions need a human in the loop. But manual review bottlenecks are exactly what kills automation ROI. As the AI Human-in-the-Loop Orchestration Engineer, you'll build the orchestration layer that intelligently routes tasks between automated AI decisions and human reviewers. You'll design confidence thresholds, escalation logic, reviewer queue management, and the audit trail infrastructure that regulators require. You'll also build feedback mechanisms that let human decisions improve model confidence over time. This is a senior role with significant system design responsibility. You'll collaborate with ML engineers who own the underlying models and with enterprise clients who define the compliance requirements. We run a mature engineering process with strong code review culture and documented architecture decision records.

Responsibilities

  • Design and implement task routing logic between automated AI decisions and human review queues
  • Build confidence threshold and escalation frameworks
  • Develop reviewer queue management and SLA enforcement systems
  • Own the audit trail and decision logging infrastructure
  • Build feedback loops that surface human decision signals back to model training pipelines

Requirements

  • 5+ years software or ML infrastructure engineering
  • Experience designing workflow orchestration systems
  • Strong Python skills with FastAPI and Airflow
  • Familiarity with audit and compliance requirements in regulated industries
  • System design experience with a focus on reliability and explainability

Benefits

  • Fully remote
  • Equity in a profitable, growing company
  • Health, dental, and vision
  • $4,000 annual professional development budget
  • Flexible PTO policy

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

Automation & AI Workflows MLOps & AI Infrastructure

Skills

python airflow fastapi docker kubernetes

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Principal Engineer

RemoteAI company

Full-time Principal

AI Resident UK/EMEA

RemoteAI company

Full-time

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

AI Engineer

RemoteAI company

Full-time

Staff Application Security Engineer

RemoteAI company

Full-time Staff

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$130k – $160k / year

AI Expertise

Automation & AI Workflows MLOps & AI Infrastructure

Skills

python airflow fastapi docker kubernetes

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile