Profile cover
C

AI Code Sample Validation Pipeline Developer

Charlotte Tran

Full-time · Job · New_York

About the role

API documentation lives or dies on the quality of its code samples. A code sample that doesn't run, imports a deprecated package, uses an API that was renamed in the last release, or handles errors in a way that was valid two versions ago teaches developers exactly the wrong thing. Outdated code samples are one of the top frustrations developers cite when evaluating API documentation quality. We generate AI-assisted developer documentation for 55 API platforms. Code sample validation is the quality gate that keeps our generated samples current and correct. We have manual review by documentation engineers. We don't have automated execution testing that runs samples against live sandbox environments, version compatibility checking that flags samples using deprecated API surface, and regression detection that surfaces previously-passing samples that break after an API update. You'll build the code sample validation pipeline: sandboxed execution environment that runs samples in isolated containers, version compatibility analysis against the API changelog, and regression monitoring that alerts the documentation team when a previously-passing sample starts failing. The pipeline needs to run against 8 programming languages across 55 API clients.

Responsibilities

  • Build sandboxed execution pipeline running code samples in isolated containers per programming language
  • Implement version compatibility analysis flagging samples using deprecated or renamed API surface
  • Develop regression monitoring detecting previously-passing samples that fail after API updates
  • Create failure classification distinguishing syntax errors, deprecated calls, runtime errors, and auth issues
  • Build execution result database tracking sample correctness history across API versions
  • Maintain language runtime environments ensuring sandbox versions match developer target environments

Requirements

  • 3+ years in developer tooling, documentation engineering, or CI/CD pipeline development
  • Strong Python and Docker skills with multi-language code execution environments
  • Understanding of API versioning, deprecation patterns, and SDK release management
  • Experience building automated testing pipelines for code correctness verification
  • Attention to sandbox security and isolation in code execution environments

Benefits

  • Fully remote with flexible schedule
  • Health and dental insurance
  • $3,000 annual learning budget
  • 401(k) with match
  • Equity options

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

AI Product & Strategy MLOps & AI Infrastructure

Skills

python docker sql fastapi openai api

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Product Manager, Developer Experience

RemoteAI company

Full-time

AI Resident UK/EMEA

RemoteAI company

Full-time

Devops Engineer, US

RemoteAI company

Full-time

Infrastructure Engineer

RemoteAI company

Full-time

PhD Autonomy Engineer Intern - Computer Vision / Deep Learning Summer 2027

RemoteAI company

Full-time Intern

Job Type

Full-time

Level

Mid-Level

Language

English

Salary Range

$100k – $130k / year

AI Expertise

AI Product & Strategy MLOps & AI Infrastructure

Skills

python docker sql fastapi openai api aws

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile