Rubric Questions Expert (Generalist)
Contract: Hourly • Location: Remote • Compensation: $45/hour
Company: Mercor • Openings: 20 • Apply by: Monday, September 8, 2025
Help improve how AI systems respond to user-generated images and prompts. You’ll design clear, detailed rubrics that guide the evaluation of AI-generated outputs—so models learn to be more accurate, helpful, and safe.
Write thorough, unambiguous rubrics for assessing AI responses to user-uploaded images and prompts.
Define scoring criteria, edge cases, and examples that make expectations crystal clear.
Review sample outputs to validate rubric quality and consistency.
Collaborate with project leads to iterate on guidelines and resolve ambiguities.
Maintain high throughput while upholding accuracy and consistency standards.
Exceptional written communication and attention to detail.
Proven experience with rubric design, evaluation criteria, technical/UX writing, data annotation, or similar.
Ability to break complex tasks into objective, measurable steps.
Comfort working independently, managing deadlines, and documenting decisions.
Familiarity with AI/ML, prompt writing, or content evaluation is a plus.
Direct impact on real-world AI behavior and quality.
Flexible, fully remote work with steady, hourly pay.
Clear scope, practical deliverables, and supportive project leadership.
Upload resume → AI interview → Complete form → Submit application
Upload your best, up-to-date resume in English. Mercor will extract details and fill out your profile automatically. Review and adjust as needed.
SHOCKING FACT: Only ~20% of applicants complete their application! Take the 15-minute AI interview about your experience and you'll have MUCH HIGHER chances of getting hired!
AI Interview Tips: The interview focuses on your resume and work experience. Be ready to discuss specific projects and how you solved challenges.
Takes about 15 minutes | Dramatically improves your chances