AI Toxic Content Detection Researcher

Yovex Intelligence

Remote · $80k – $100k / year · Full-time · Posted 4 months ago

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

This role is no longer accepting applications

Don't miss the next role like this

Create a free profile with your skills. We match you to new AI roles as they open, and companies hiring can contact you directly.

Create free profile, 2 minutes

Similar open roles you can apply to now:

See all open AI jobs

About the role

We don't care where you work from. We care about the quality of your thinking, your ability to read papers critically, and whether you can ship models that make online spaces safer. Yovex Intelligence builds trust and safety infrastructure for social platforms and AI products, and toxic content detection is one of our core research tracks. In this junior research role, you'll work alongside our senior safety researchers to build, evaluate, and improve toxic content classifiers across categories including hate speech, harassment, and self-harm promotion. You'll implement recent published methods, run comparative evaluations across datasets with different demographic distributions, and help maintain the annotation guidelines used for training data labeling. This is a research-leaning role accessible to a strong junior candidate. We expect familiarity with Hugging Face, a solid grasp of classification evaluation metrics, and the intellectual honesty to surface when a model underperforms on a specific demographic group. Safety research requires rigor, not just accuracy.

Fully remote.

Responsibilities

Implement and evaluate toxic content classifiers across multiple harm categories
Run comparative evaluations across datasets with different demographic distributions
Maintain annotation guidelines used for training data labeling
Assist in literature reviews of recent content moderation and safety research
Document model limitations and demographic performance gaps clearly

Requirements

Basic Python and PyTorch skills
Familiarity with Hugging Face transformers
Understanding of classification evaluation metrics (F1, AUC, demographic parity)
Interest in AI safety, content moderation, or trust and safety research
Strong analytical thinking and intellectual honesty about model failures