AI Code Sample Validation Pipeline Developer

Navio Tech

Remote · $100k – $130k / year · Full-time · Posted 4 months ago

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

API documentation lives or dies on the quality of its code samples. A code sample that doesn't run, imports a deprecated package, uses an API that was renamed in the last release, or handles errors in a way that was valid two versions ago teaches developers exactly the wrong thing. Outdated code samples are one of the top frustrations developers cite when evaluating API documentation quality. We generate AI-assisted developer documentation for 55 API platforms. Code sample validation is the quality gate that keeps our generated samples current and correct. We have manual review by documentation engineers. We don't have automated execution testing that runs samples against live sandbox environments, version compatibility checking that flags samples using deprecated API surface, and regression detection that surfaces previously-passing samples that break after an API update. You'll build the code sample validation pipeline: sandboxed execution environment that runs samples in isolated containers, version compatibility analysis against the API changelog, and regression monitoring that alerts the documentation team when a previously-passing sample starts failing. The pipeline needs to run against 8 programming languages across 55 API clients.

Fully remote.

Responsibilities

Build sandboxed execution pipeline running code samples in isolated containers per programming language
Implement version compatibility analysis flagging samples using deprecated or renamed API surface
Develop regression monitoring detecting previously-passing samples that fail after API updates
Create failure classification distinguishing syntax errors, deprecated calls, runtime errors, and auth issues
Build execution result database tracking sample correctness history across API versions
Maintain language runtime environments ensuring sandbox versions match developer target environments

Requirements

3+ years in developer tooling, documentation engineering, or CI/CD pipeline development
Strong Python and Docker skills with multi-language code execution environments
Understanding of API versioning, deprecation patterns, and SDK release management
Experience building automated testing pipelines for code correctness verification
Attention to sandbox security and isolation in code execution environments