Rubric Questions Expert (Generalist) at Mercor

Rubric Questions Expert (Generalist)

Contract: Hourly • Location: Remote • Compensation: $45/hour
Company: Mercor • Openings: 20 • Apply by: Monday, September 8, 2025

About the Project

Help improve how AI systems respond to user-generated images and prompts. You’ll design clear, detailed rubrics that guide the evaluation of AI-generated outputs—so models learn to be more accurate, helpful, and safe.

What You’ll Do

• Write thorough, unambiguous rubrics for assessing AI responses to user-uploaded images and prompts.

• Define scoring criteria, edge cases, and examples that make expectations crystal clear.

• Review sample outputs to validate rubric quality and consistency.

• Collaborate with project leads to iterate on guidelines and resolve ambiguities.

• Maintain high throughput while upholding accuracy and consistency standards.

What You’ll Bring

• Exceptional written communication and attention to detail.

• Proven experience with rubric design, evaluation criteria, technical/UX writing, data annotation, or similar.

• Ability to break complex tasks into objective, measurable steps.

• Comfort working independently, managing deadlines, and documenting decisions.

• Familiarity with AI/ML, prompt writing, or content evaluation is a plus.

Why Join

• Direct impact on real-world AI behavior and quality.

• Flexible, fully remote work with steady, hourly pay.

• Clear scope, practical deliverables, and supportive project leadership.