Profile cover
O

AI Multi-Model Routing Architect

Emrex AI

Remote · $160k – $200k / year · Full-time · Posted 4 months ago

No longer accepting applications

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

Fast-moving team. Big problem. We need someone who ships. Emrex AI runs inference across a portfolio of 12+ language models — ranging from small, fast, cheap local models to large frontier APIs — and the routing logic that decides which request goes where is currently hand-coded rules that break every time we add a new model. You'll redesign that routing layer as a learned system: a classifier that predicts the optimal model for each request based on task complexity, latency budget, cost constraints, and quality requirements. You'll build the evaluation harness that continuously measures model-task fit, and you'll maintain the online learning pipeline that keeps routing decisions current as model capabilities shift. This is a technical architecture role with direct product impact. Every routing improvement translates to measurable cost reduction or quality improvement across 40M+ inference calls per day. We run weekly architecture reviews and expect this engineer to drive the technical roadmap for the routing layer.

Fully remote.

Responsibilities

  • Design and implement a learned routing system for multi-model LLM inference
  • Build the evaluation harness that measures model-task fit across routing decisions
  • Maintain online learning pipelines that adapt routing as model capabilities change
  • Partner with ML teams to define task complexity signals and quality metrics
  • Own the routing layer SLA and incident response

Requirements

  • 5+ years ML infrastructure or applied ML engineering experience
  • Strong Python skills with LangChain and FastAPI
  • Experience with LLM inference pipelines at scale
  • Familiarity with multi-armed bandit or contextual bandit approaches is a plus
  • Ability to own a technical domain end-to-end with minimal supervision

Benefits

  • Fully remote
  • Top-of-market compensation with equity
  • $8,000 annual GPU compute credits for personal research
  • Health, dental, and vision coverage
  • Quarterly engineering offsite

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python pytorch langchain openai api fastapi

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

AI Researcher - Pre-training (f/m/d)

Aleph Alpha

Full-time

Research Engineer / Scientist, Pre-training

Anthropic

Full-time

Lead/ Senior Machine Learning Scientist, Integrative Phenotyping

insitro

Full-time Lead

AI Security Engineer Intern (Summer 2027)

Brainco

Full-time Intern

AWS Alliances Lead

LangChain

Full-time Lead

AWS Alliances Lead

LangChain

Full-time Lead

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

MLOps & AI Infrastructure AI & Machine Learning Engineers

Skills

python pytorch langchain openai api fastapi

Ready to apply for this role?

Create a free talent account.

  • Find AI roles in one place
  • Get AI-matched job recommendations
  • Reply to companies that message you
  • Build your public AI talent profile
Create free account & apply Log in

This role is closed. Create your free profile to apply to similar open roles.

Don't miss the next one: get matched to new roles like this.

Create free profile