Profile cover
A

AI Emergent Capability Probe Engineer

Aria Watanabe

Full-time · Job · San_Francisco

About the role

Emergent capabilities — behaviors that appear in large language models at scale without being explicitly trained — are one of the most poorly understood phenomena in AI development. They matter for safety because a capability that isn't known to exist can't be evaluated for misuse potential or monitored in deployment. The goal of capability probing is to systematically discover what a model can do before deployment, not after. We're an AI safety research organization that develops evaluation methodology for frontier models. Our capability probing work involves designing elicitation prompts that reliably surface latent capabilities, building automated discovery pipelines that search for unexpected model behaviors, and creating standardized assessments that can track capability emergence as a function of model scale. This is a senior research engineer role at the intersection of ML and AI safety evaluation. You'll design probing methodologies for capability categories we care about — deceptive alignment, situational awareness, strategic reasoning, novel tool use — build the automated probing infrastructure, and document your findings in research-quality reports that inform our policy recommendations and model evaluation partnerships. The role is hybrid, with 2-3 days per week in our SF office for research team collaboration.

Responsibilities

  • Design systematic elicitation methodologies for probing specific model capability categories
  • Build automated capability discovery pipelines searching for unexpected model behaviors at scale
  • Implement quantitative capability benchmarks enabling cross-model and cross-scale comparison
  • Develop probing evaluation methodology robust to prompt sensitivity and elicitation variance
  • Document capability findings in research reports informing safety evaluation frameworks
  • Collaborate with alignment researchers to prioritize capability categories based on risk relevance

Requirements

  • 6+ years in ML research with model evaluation, AI safety, or interpretability experience
  • Deep understanding of LLM capabilities and their relationship to scale, training data, and RLHF
  • Strong Python and experimental design skills
  • Research track record with publications or significant contributions in AI evaluation or safety
  • Ability to design rigorous empirical studies with appropriate controls and statistical analysis

Benefits

  • Hybrid: 2-3 days SF office, remainder remote
  • Top-of-market compensation for AI safety research expertise
  • Research publication and conference budget
  • Comprehensive health benefits
  • Mission-aligned equity participation

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

AI & Machine Learning Engineers AI Product & Strategy

Skills

python pytorch hugging face statistics spark

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior AI Researcher- Pre-training (f/m/d)

RemoteAI company

Full-time Senior

AI Resident UK/EMEA

RemoteAI company

Full-time

Product Manager, Developer Experience

RemoteAI company

Full-time

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

Senior Machine Learning Engineer (Clinical Team)

RemoteAI company

Full-time Senior

Conversational Modelling Research Engineer

RemoteAI company

Full-time

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

AI & Machine Learning Engineers AI Product & Strategy

Skills

python pytorch hugging face statistics spark sql

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile