Posted over 30 days ago. Roles like this often stay open for a while, so it's still worth applying.
JSON & Rubrics Grading Expert at Mercor
posted 3 months agoJSON & Rubrics Grading Expert | $50–$175/hr | Remote (US Only)
Mercor is seeking detail-oriented graders for an immediate weekend sprint starting July 24th. This is a short-term contract role supporting one of the world's leading AI research organizations. You will evaluate AI model outputs against structured rubrics — no AI tools permitted.
What You'll Do
- Read and interpret written task descriptions and their associated grading rubrics
- Parse and analyze JSON-formatted AI model outputs
- Grade JSON content consistently and accurately against provided rubric criteria
- Complete all assessments independently, without the use of AI tools
Requirements
- Available to begin work immediately on July 24th, running through July 27–28 (with potential extension)
- Comfortable reading and interpreting JSON files
- Ability to dedicate 20–40 hours per week to this project
- Able to turn around tasks within 12–24 hours
- Access to a desktop or laptop computer (Chromebooks not accepted)
- Based in the United States
Nice to Have
- Prior experience with Mercor projects
- Background in applying detailed rubrics and evaluation criteria
Attention to detail and consistency are essential. Apply only if you can start today!
How to apply for this role
- Upload your resume — keep it up-to-date and in English. Mercor will auto-fill your profile from it.
- Complete the AI interview — a 15-minute conversation about your experience. Be ready to discuss specific projects and challenges you've solved.
- Submit your application — only about 20% of applicants finish all the steps, so completing yours puts you well ahead.