AI Training Scenarios Reviewer (QC) at Mercor

AI Training Scenarios Reviewer (QC) | $80–105/hr | Remote (US)

Join a cutting-edge team as a Quality Control Reviewer for AI training scenarios. You'll audit and evaluate complex simulated digital environments designed to train frontier AI agents, ensuring every scenario meets the highest standards of realism, logic, and technical accuracy. This is a W-2 full-time position (40 hrs/week) through Cincinnatus LLC, embedded within a leading AI lab's extended workforce.

What You'll Do

• Audit & Evaluate: Review detailed personas, digital environments (Gmail, Slack, Drive, etc.), and agentic tasks created by AI Training Scenario Designers for logical soundness and structural integrity.
• Apply Grading Rubrics: Score scenarios objectively using comprehensive evaluation rubrics, ensuring tasks effectively challenge AI reasoning without being broken or impossible.
• Quality Assurance: Identify logical inconsistencies, factual errors, missing context, and technical flaws such as invalid JSON structures.
• Provide Feedback: Deliver clear, constructive, and actionable feedback to scenario designers to improve their output.
• Document Outcomes: Log review scores and feedback rigorously in tools like Airtable and Crucible.

You're a Great Fit If You

• Have a proven background in QA, editing, auditing, or reviewing complex written or technical content.
• Excel at learning and applying detailed grading rubrics — prior high scores in rubric-based training is a strong plus.
• Have an eagle eye for inconsistencies, edge cases, and logical gaps.
• Are comfortable giving structured, constructive feedback to peers and creators.
• Can navigate tools like JSON editors, Airtable, Jira, or similar platforms with ease.

Requirements

• Bachelor's degree (required)
• 2+ years of professional experience (required)
• Availability for 40 hours/week

Nice to Have

• Editing/QA background: Senior editor, QA tester, technical reviewer, compliance auditor, or peer-reviewer experience.
• Technical comfort: JSON familiarity, SQL/Python exposure, or power-user experience with Jira/Asana. You don't need to write code — but you must spot a broken JSON structure.
• Systems thinking: Operations managers, technical PMs, or systems designers who understand complex, multi-stakeholder workflows.

About Cincinnatus LLC

Cincinnatus LLC is an enterprise staffing company partnering with leading technology firms to place highly skilled professionals in structured, role-based contract positions. As the employer of record, Cincinnatus provides W-2 employment, payroll, benefits, and compliance while integrating employees directly into client teams. This is not a freelance or project-based engagement — it is a full-time, structured role with close collaboration with an AI lab's internal teams.

Cincinnatus LLC is an Equal Employment Opportunity employer and is committed to providing reasonable accommodations for qualified individuals with disabilities.