This job post has expired on September 29, 2025. It is likely that the position has already been filled.
Rubric Questions Expert (Generalist) at Mercor
posted 1 year agoContract: Hourly • Location: Remote • Compensation: $45/hour
Company: Mercor • Openings: 20 • Apply by: Monday, September 8, 2025
About the Project
Help improve how AI systems respond to user-generated images and prompts. You’ll design clear, detailed rubrics that guide the evaluation of AI-generated outputs—so models learn to be more accurate, helpful, and safe.
What You’ll Do
-
Write thorough, unambiguous rubrics for assessing AI responses to user-uploaded images and prompts.
-
Define scoring criteria, edge cases, and examples that make expectations crystal clear.
-
Review sample outputs to validate rubric quality and consistency.
-
Collaborate with project leads to iterate on guidelines and resolve ambiguities.
-
Maintain high throughput while upholding accuracy and consistency standards.
What You’ll Bring
-
Exceptional written communication and attention to detail.
-
Proven experience with rubric design, evaluation criteria, technical/UX writing, data annotation, or similar.
-
Ability to break complex tasks into objective, measurable steps.
-
Comfort working independently, managing deadlines, and documenting decisions.
-
Familiarity with AI/ML, prompt writing, or content evaluation is a plus.
Why Join
-
Direct impact on real-world AI behavior and quality.
-
Flexible, fully remote work with steady, hourly pay.
-
Clear scope, practical deliverables, and supportive project leadership.
How to apply for this role
- Upload your resume — keep it up-to-date and in English. Mercor will auto-fill your profile from it.
- Complete the AI interview — a 15-minute conversation about your experience. Be ready to discuss specific projects and challenges you've solved.
- Submit your application — only about 20% of applicants finish all the steps, so completing yours puts you well ahead.