Data Annotator

Role Overview

Mercor is partnering with a top AI lab on a project to train and assess the ability of LLMs to evaluate video-based content. We’re looking for detail-oriented contributors to help create question-and-answer pairs and evaluation rubrics. This is a flexible opportunity that can fit any schedule.

Key Responsibilities

• Review short and long-form videos to create question-and-answer (QA) pairs.
• Develop rubric-based evaluation questions and answers to assess multiple dimensions of model output.
• Apply structured guidelines to ensure high-quality evaluations.

Ideal Qualifications

• Experience in annotation, data labeling, or related content evaluation work.
• Strong English writing skills and ability to craft clear, specific questions and answers.
• Comfortable working independently and managing flexible tasks with attention to detail.
• General familiarity with multimedia content (e.g., videos from social media, how-tos, documentaries).

More About the Opportunity

• Remote and asynchronous.
• Expected commitment: 30-40 hours per week

Compensation & Contract Terms

• Applicants must be based in the US or Canada.
• This position is structured with clear expectations.
• Payments issued weekly via Stripe Connect

Application Process

• Submit your resume to begin
• After submitting your resume, please complete a short AI interview and form.
• Follow-up typically occurs within 3–5 business days

About Mercor

• Mercor is a talent marketplace that connects top experts with leading AI labs and research organizations.
• Our investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey.