AI Context Window Optimization Engineer

Lontex Tech

Remote · $160k – $200k / year · Full-time · Posted 5 months ago

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

LLM context windows keep getting longer, but throwing 200K tokens at every query is like using a firehose to water a plant. Our platform helps enterprises use context efficiently — retrieval-augmented generation, dynamic context compression, attention steering, and smart caching — so they get better answers at 1/10th the token cost. The research is clear on what works. The engineering challenge is making it work reliably across 50+ client deployments with different document types, query patterns, and latency requirements. You'll build the context optimization middleware that sits between our clients' applications and their LLM providers, automatically applying the right compression and retrieval strategy based on the query type. You'll be the fifth engineer on the core platform team. We ship weekly to production, and every feature gets measured on two axes: answer quality (judged by automated evals) and token cost reduction. If quality drops, the feature gets reverted. No exceptions.

Fully remote.

Responsibilities

Build context optimization middleware for enterprise LLM deployments
Implement dynamic compression strategies based on query classification
Design and maintain RAG pipelines optimized for different document types
Create automated evaluation frameworks for answer quality measurement
Profile and reduce token costs while maintaining output quality
Support client integration and troubleshoot deployment-specific issues

Requirements

5+ years in NLP or ML engineering with LLM production experience
Deep understanding of RAG architectures, embedding models, and retrieval strategies
Experience with context window management and token optimization
Strong Python and system design skills
Ability to measure and optimize for competing objectives: quality vs cost