Profile cover
S

AI Long-Context Chunking Research Engineer

Stella Tran

Full-time · Job · San_Francisco

About the role

Context windows are getting longer. 128K, 200K, 1M tokens. The naive conclusion is that chunking becomes less important — just put everything in context. The sophisticated conclusion is that chunking strategy becomes more important, not less: with 1M tokens in context, the model's attention is spread across 800,000 tokens of potentially irrelevant content, and the signal-to-noise ratio of the relevant passage plummets. Long-context retrieval that doesn't chunk intelligently is worse than short-context retrieval that chunks well. We build LLM serving and retrieval infrastructure handling 140 million API calls per month. Long-context retrieval strategy is an active research area for us, and we need a senior research engineer who can push the state of the art for our production use cases. The question we're trying to answer is: for documents too large to fit in any context window, and for queries that require multi-hop reasoning across distributed document sections, what chunking and retrieval strategy minimizes information loss? You'll lead this research program: designing experiments, implementing novel chunking approaches, building evaluation infrastructure, and translating findings that generalize into production retrieval improvements. Expect to publish.

Responsibilities

  • Design and implement novel chunking approaches for long-document retrieval with empirical evaluation
  • Build multi-hop retrieval strategies for queries requiring information synthesis across document sections
  • Develop information-theoretic chunking evaluation measuring what each strategy preserves versus loses
  • Implement context window utilization analysis identifying the effective retrieval threshold per model family
  • Create production retrieval benchmarks comparing long-context versus chunked retrieval strategies
  • Publish research findings in LLM systems or information retrieval venues

Requirements

  • 6+ years in ML research with retrieval systems, LLM applications, or NLP engineering
  • Deep understanding of attention mechanisms, context window behavior, and RAG architectures
  • Strong Python and PyTorch skills with information retrieval evaluation methodology
  • Research track record with publications or significant contributions in LLM systems or IR
  • Ability to design rigorous experiments that distinguish genuine improvement from benchmark artifacts

Benefits

  • Fully remote with flexible schedule
  • Top-of-market compensation for LLM research expertise
  • Research conference and publication budget
  • Comprehensive health benefits
  • Equity participation

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

AI & Machine Learning Engineers MLOps & AI Infrastructure

Skills

python pytorch hugging face rag docker statistics

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile

Interested in this role? Create your free profile to apply.

Apply — create free profile