Profile cover
S

AI Context Compression Research Engineer

Stella Tran

Full-time · Job · New_York

About the role

Context windows are getting longer, but the fundamental economics remain unchanged: longer contexts cost more per token, and most tokens in a long context do not meaningfully contribute to the quality of the response. Context compression — identifying and retaining the semantically critical subset of a long context — can reduce costs by 60 to 80% without degrading response quality in the use cases that actually matter. We build LLM serving infrastructure for enterprise clients. Our context compression layer sits between the application and the model, analyzing incoming contexts and applying learned compression strategies that preserve decision-relevant information while discarding redundant or irrelevant content. The challenge is that what is relevant depends on the query — compression must be query-conditioned, not static. You will lead the compression research program: designing query-conditioned compression architectures, building evaluation benchmarks that measure quality preservation across different task types, and developing the calibration approach that determines per-task compression aggressiveness.

Responsibilities

  • Design query-conditioned context compression architectures
  • Build evaluation benchmarks measuring quality preservation across task types
  • Develop calibration approach for per-task compression aggressiveness tuning
  • Implement compression pipeline with sub-50ms overhead for real-time serving
  • Research and adapt recent work on context distillation and selective attention
  • Publish research findings on context compression methodology

Requirements

  • 5+ years in ML research or engineering with NLP and LLM systems experience
  • Deep understanding of transformer attention mechanisms and context utilization
  • Strong PyTorch skills and experience with LLM serving infrastructure
  • Experience designing evaluation benchmarks for generative AI quality
  • Track record of research publications or significant open-source contributions

Benefits

  • Fully remote
  • Research conference and publication budget
  • Top-of-market compensation
  • Health, dental, and vision
  • Equity with 4-year vesting

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

AI & Machine Learning Engineers NLP & Prompt Engineering

Skills

python pytorch hugging face statistics docker

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Senior AI Researcher- Pre-training (f/m/d)

RemoteAI company

Full-time Senior

Drone Platform Integration & Network Support Engineer

RemoteAI company

Full-time

AI Resident UK/EMEA

RemoteAI company

Full-time

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

Senior Machine Learning Engineer (Clinical Team)

RemoteAI company

Full-time Senior

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

AI & Machine Learning Engineers NLP & Prompt Engineering

Skills

python pytorch hugging face statistics docker aws

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile