This job post has expired on June 30, 2025. It is likely that the position has already been filled.
Generalist Video Evaluation Expert
Location: Remote (Applicants must be based in the US or Canada)
Compensation: $45.50/hour (Hourly contract, payments made weekly via Stripe Connect)
Role Overview
Mercor is partnering with a leading AI lab to enhance Large Language Models (LLMs) in evaluating video content effectively. We seek detail-oriented contributors to help develop structured question-and-answer pairs and precise evaluation rubrics. This flexible role can easily accommodate various schedules.
Key Responsibilities
Review short-form and long-form videos to create insightful and relevant question-and-answer (QA) pairs.
Design and refine rubric-based evaluation questions to thoroughly assess model outputs.
Consistently apply established guidelines to ensure quality and accuracy of evaluations.
Ideal Qualifications
Previous experience in annotation, data labeling, content evaluation, or related tasks.
Exceptional English writing skills, with the ability to create clear, precise, and targeted QA pairs.
Comfortable working independently, demonstrating strong attention to detail and time management.
Familiarity with various multimedia formats, including social media videos, instructional videos, and documentaries.
Additional Details
Fully remote and asynchronous work arrangement.
Expected commitment: 30-40 hours per week.
Application Process
Submit your resume to initiate your application.
Complete a short AI-powered interview and an additional brief form.
Expect a response within 3–5 business days.
About Mercor
Mercor connects exceptional talent with prominent AI labs and research organizations. Our investors include industry leaders from Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey.
Apply today to become part of our cutting-edge project to shape the future of AI video evaluation.