Profile cover
O

AI Multi-Model Routing Architect

Owen Goldberg

Full-time · Job · Los_Angeles

About the role

Fast-moving team. Big problem. We need someone who ships. Emrex AI runs inference across a portfolio of 12+ language models — ranging from small, fast, cheap local models to large frontier APIs — and the routing logic that decides which request goes where is currently hand-coded rules that break every time we add a new model. You'll redesign that routing layer as a learned system: a classifier that predicts the optimal model for each request based on task complexity, latency budget, cost constraints, and quality requirements. You'll build the evaluation harness that continuously measures model-task fit, and you'll maintain the online learning pipeline that keeps routing decisions current as model capabilities shift. This is a technical architecture role with direct product impact. Every routing improvement translates to measurable cost reduction or quality improvement across 40M+ inference calls per day. We run weekly architecture reviews and expect this engineer to drive the technical roadmap for the routing layer.

Responsibilities

  • Design and implement a learned routing system for multi-model LLM inference
  • Build the evaluation harness that measures model-task fit across routing decisions
  • Maintain online learning pipelines that adapt routing as model capabilities change
  • Partner with ML teams to define task complexity signals and quality metrics
  • Own the routing layer SLA and incident response

Requirements

  • 5+ years ML infrastructure or applied ML engineering experience
  • Strong Python skills with LangChain and FastAPI
  • Experience with LLM inference pipelines at scale
  • Familiarity with multi-armed bandit or contextual bandit approaches is a plus
  • Ability to own a technical domain end-to-end with minimal supervision

Benefits

  • Fully remote
  • Top-of-market compensation with equity
  • $8,000 annual GPU compute credits for personal research
  • Health, dental, and vision coverage
  • Quarterly engineering offsite

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python pytorch langchain openai api fastapi

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior AI Researcher- Pre-training (f/m/d)

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

Senior Machine Learning Engineer (Clinical Team)

RemoteAI company

Full-time Senior

Conversational Modelling Research Engineer

RemoteAI company

Full-time

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python pytorch langchain openai api fastapi

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile