Legal Rubric & Ground Truth Auditor (Attorney) at Mercor

Licensed Attorney | $120-140/hr | Remote (US) | Short-term contract

Mercor is seeking experienced licensed attorneys to audit rubrics and ground-truth answers for a legal research dataset used to evaluate advanced AI systems. You will review existing legal research questions, verify the accuracy of model answers, and refine grading rubrics so the dataset reflects current law and sound legal reasoning. This is a short-term, fully remote engagement.

What the work involves

• Audit grading rubrics for accuracy, completeness, consistency and clarity.
• Verify ground-truth answers against controlling statutes, regulations and case law.
• Identify legal errors, outdated authority, missing issues and ambiguous grading criteria.
• Rewrite or correct ground-truth answers and rubric items where needed.
• Provide concise, structured written feedback on each item reviewed.

Practice areas

The dataset spans eight practice areas. Experience in at least one is required, and breadth across several is a plus: administrative/regulatory, business & commercial, civil litigation, constitutional/civil rights, criminal, family, health and immigration.

Requirements

• Juris Doctor (JD) and active U.S. bar admission in good standing.
• 5+ years of legal practice.
• Substantive experience in at least one of the practice areas above.
• Strong legal research, analysis and writing skills.
• High attention to detail and the ability to apply consistent judgment.

Preferred qualifications

• Experience across multiple listed practice areas.
• Judicial clerkship, appellate, government or law firm research experience.
• Prior experience reviewing or grading legal work product, or with AI or legal-tech evaluation projects.

Pay and location

$120-140/hr as a contractor. The work is fully remote and restricted to applicants in the United States. This is a short-term engagement.