Professional Overview
I specialize in AI/LLM evaluation and coding-agent benchmarking, backed by 20+ years of production software engineering. Across Mercor, Snorkel AI, Aligneer, Handshake, Outlier, Turing, and Shift, my work includes evaluation harnesses, reference solutions, regression tests, grading rubrics, RLHF preference annotation, and multimodal data annotation. I build repository-based coding tasks with Docker environments and automated verifiers, then analyze agent trajectories for correctness, regressions, reward hacking, and answer leakage. In a Java coding-agent evaluation, I diagnosed API and state-integrity failures, refined task instructions, and independently verified the corrected implementation with 24 passing tests. My technical background includes Java, Python, SQL, TypeScript, React, AWS, and CI/CD, with production systems supporting 5M+ monthly requests and 99.9% uptime. I bring practical engineering judgment to model evaluation, benchmark design, and AI training data quality.
Experience
AI Expertise
Skills
Employment Type
Description of Responsibilities
- Build Snorkel AI Harbor tasks with golden patches and held-out tests
- Write golden solutions and regression tests from real GitHub PRs for Handshake
- Score Aligneer coding-agent trajectories on 7 quality dimensions in Claude Code
- Write Turing red-team prompts to break Gemini, Claude, and Qwen on reasoning
- Annotate robot-learning video for Shift with contact boundaries at ±5 frames
Key Achievements / KPIs
- Shipped 5 Snorkel tasks: oracle passed 3/3 runs, no-op agent scored 0 on each
- Grew graded tests on a Snorkel task from 13 to 34
- Built two tasks where Claude Opus 4.8 and Codex GPT-5.5 both scored 0/8
- Wrote 35-55 item grading rubrics for Handshake tasks without LLM help
AI Expertise
Skills
Employment Type
Description of Responsibilities
- Grade Claude Code output on Java/DDD refactoring tasks with hidden rubrics, gold
- Rank up to 6 AI-generated spreadsheets, slide decks, and documents per task agai
- Write coding prompts calibrated to 0-40% and 40-70% AI pass-rate bands for vibe-
- Produce RLHF preference labels from paired model outputs for a phased frontier-m
- Audit an AI agent’s Excel analysis of recruiting data for scoring anomalies, dup
Key Achievements / KPIs
- Raised a Claude Code run from 20 to 24 passing tests by tracing two defects to p
- Ranked AI deliverables across 52 tasks: 29 spreadsheet tasks and 23 slide-deck t
- Passed the Mercor Spreadsheet Proficiency exam 12/12, proctored, with AI tools p
- Completed 11 separate Mercor engagements since 2025 across grading, ranking, RLH
AI Expertise
Skills
Employment Type
Description of Responsibilities
- Built fleet services for remote updates on thousands of IoT payment terminals
- Wrote C++ and Java code for low-latency TCP/IP links to terminal firmware
- Designed Kafka pipelines for terminal telemetry and deployment orchestration
- Provisioned AWS, Azure, and GCP with Terraform, mainly AWS Lambda and Aurora
- Integrated Angular frontends with Node.js/Express services and MongoDB
Key Achievements / KPIs
- Cut deployment time 60% with Jenkins and GitLab CI/CD, enabling daily releases
- Kept 99.9% uptime on IoT device services with tuned PostgreSQL and Flyway
- Cut incident triage time with Prometheus, Loki, and Grafana fleet dashboards
- Added SonarQube, Black Duck, and Fortify security scans to CI/CD pipelines
Education
Field of Study: Machine Learning and Pattern Recognition
Field of Study: Operations Research and Linear Programming (Simplex Method)
Field of Study: Mathematics with Computer Science
Areas of Expertise
NLP & Prompt Engineering
Build natural language processing systems, fine-tune large language models, and craft reliable production prompts for AI-powered products.
MLOps & AI Infrastructure
Architect scalable ML infrastructure — CI/CD for ML, model versioning, experiment tracking, monitoring, and cloud deployment on AWS, GCP, or Azure.
Automation & AI Workflows
Build intelligent automation pipelines, integrate AI APIs into existing tools, and replace manual workflows with agentic AI systems.
