This job post has expired on September 29, 2025. It is likely that the position has already been filled.
Rubric Questions Expert (Generalist)
Contract: Hourly • Location: Remote • Compensation: $45/hour
Company: Mercor • Openings: 20 • Apply by: Monday, September 8, 2025
Help improve how AI systems respond to user-generated images and prompts. You’ll design clear, detailed rubrics that guide the evaluation of AI-generated outputs—so models learn to be more accurate, helpful, and safe.
Write thorough, unambiguous rubrics for assessing AI responses to user-uploaded images and prompts.
Define scoring criteria, edge cases, and examples that make expectations crystal clear.
Review sample outputs to validate rubric quality and consistency.
Collaborate with project leads to iterate on guidelines and resolve ambiguities.
Maintain high throughput while upholding accuracy and consistency standards.
Exceptional written communication and attention to detail.
Proven experience with rubric design, evaluation criteria, technical/UX writing, data annotation, or similar.
Ability to break complex tasks into objective, measurable steps.
Comfort working independently, managing deadlines, and documenting decisions.
Familiarity with AI/ML, prompt writing, or content evaluation is a plus.
Direct impact on real-world AI behavior and quality.
Flexible, fully remote work with steady, hourly pay.
Clear scope, practical deliverables, and supportive project leadership.