Profile cover
O

AI Context Window Overflow Handling Engineer

Owen Goldberg

Full-time · Job · Los_Angeles

About the role

Prompt caching, RAG quality, and multi-model routing are the core of what we build. Long-running agent sessions keep hitting context window limits mid-task, and today the fallback behavior is a hard truncation that silently drops earlier instructions. You'll design a graceful overflow strategy that summarizes, re-ranks, or offloads context before the limit is hit, so agents keep working coherently on long tasks instead of failing quietly. This is a high-visibility problem affecting nearly every customer workload on our platform.

Responsibilities

  • Design overflow-handling strategies for agent sessions approaching context limits
  • Build summarization and re-ranking logic to preserve critical context under pressure
  • Benchmark overflow strategies against task completion quality on long sessions
  • Partner with applied teams to roll out overflow handling across production agents

Requirements

  • 5+ years building LLM infrastructure or agent tooling
  • Deep hands-on experience with LangChain and prompt engineering practices
  • Familiarity with RAG pipelines and context management techniques
  • Experience designing systems that fail gracefully under real-world load

Benefits

  • Fully remote position
  • Significant equity
  • Health, dental, and vision coverage
  • Conference speaking budget
  • Flexible PTO

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

NLP & Prompt Engineering MLOps & AI Infrastructure

Skills

python langchain prompt engineering openai api rag

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Software Engineer, Full Stack - UK

RemoteAI company

Full-time

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Team Lead, Actors

RemoteAI company

Full-time Lead

AI Resident UK/EMEA

RemoteAI company

Full-time

Software Engineer, Applications (Scientific Development Solutions) (High Seniority)

RemoteAI company

Full-time Senior

PhD Autonomy Engineer Intern - Computer Vision / Deep Learning Summer 2027

RemoteAI company

Full-time Intern

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

NLP & Prompt Engineering MLOps & AI Infrastructure

Skills

python langchain prompt engineering openai api rag

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile