Profile cover
S

AI Context Window Optimization Engineer

Stella Tran

Full-time · Job · Los_Angeles

About the role

LLM context windows keep getting longer, but throwing 200K tokens at every query is like using a firehose to water a plant. Our platform helps enterprises use context efficiently — retrieval-augmented generation, dynamic context compression, attention steering, and smart caching — so they get better answers at 1/10th the token cost. The research is clear on what works. The engineering challenge is making it work reliably across 50+ client deployments with different document types, query patterns, and latency requirements. You'll build the context optimization middleware that sits between our clients' applications and their LLM providers, automatically applying the right compression and retrieval strategy based on the query type. You'll be the fifth engineer on the core platform team. We ship weekly to production, and every feature gets measured on two axes: answer quality (judged by automated evals) and token cost reduction. If quality drops, the feature gets reverted. No exceptions.

Responsibilities

  • Build context optimization middleware for enterprise LLM deployments
  • Implement dynamic compression strategies based on query classification
  • Design and maintain RAG pipelines optimized for different document types
  • Create automated evaluation frameworks for answer quality measurement
  • Profile and reduce token costs while maintaining output quality
  • Support client integration and troubleshoot deployment-specific issues

Requirements

  • 5+ years in NLP or ML engineering with LLM production experience
  • Deep understanding of RAG architectures, embedding models, and retrieval strategies
  • Experience with context window management and token optimization
  • Strong Python and system design skills
  • Ability to measure and optimize for competing objectives: quality vs cost

Benefits

  • Fully remote with flexible schedule
  • Competitive salary plus meaningful equity
  • Health, dental, and vision insurance
  • $4,000 annual learning budget
  • Annual team retreat

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

AI & Machine Learning Engineers NLP & Prompt Engineering

Skills

python pytorch langchain rag openai api

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Senior AI Researcher- Pre-training (f/m/d)

RemoteAI company

Full-time Senior

Software Engineer, Full Stack - UK

RemoteAI company

Full-time

AI Resident UK/EMEA

RemoteAI company

Full-time

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

Senior Machine Learning Engineer (Clinical Team)

RemoteAI company

Full-time Senior

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

AI & Machine Learning Engineers NLP & Prompt Engineering

Skills

python pytorch langchain rag openai api docker

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile