This job post has expired on January 02, 2026. It is likely that the position has already been filled.
Rubric Academy Fellowship - AI Evaluation Expert at Mercor
posted 10 months agoAI Evaluation Expert Fellowship | $50–150 flat | Worldwide Remote
Mercor is launching the Rubric Academy, a specialized paid fellowship designed to teach subject-matter experts how to rigorously evaluate Artificial Intelligence. This is a unique opportunity to get paid while learning cutting-edge AI evaluation methodologies and unlock access to high-value expert projects.
What You Will Learn
Complete a paid, 3–5 hour interactive course teaching you the gold standard methodology of Prompt-Rubric Pairing for training and benchmarking advanced AI models:
- The Art of the Prompt: Write inputs that set clear context, clarity, and constraints to elicit specific expert behaviors from AI
- The Golden Response: Define ideal outputs beyond simple fact-checking, articulating the nuance, tone, and logic required for top-tier professional responses
- Rubric Architecture: Build objective grading systems with criterion descriptions, weights, and citations to create mathematical frameworks for judging AI performance
Who We Are Looking For
Deep domain experts across all fields—legal scholars, structural engineers, clinical psychologists, financial analysts, and more. If you have professional standards that are difficult to quantify, we want you.
- Deep Expertise: Possess unique knowledge difficult to find on the open web, with ability to cite specific sources (PDFs, datasets) supporting professional judgments
- Analytical Rigor: Detail-oriented with ability to break down complex professional tasks into specific, measurable criteria
- Integrity: Committed to original work, understanding that using AI to grade AI defeats the purpose
Project Details & Compensation
- Stipend: $50–150 flat payment upon successful completion of Academy lessons and Practice Task
- Time Commitment: Approximately 3–5 hours (self-paced)
- Future Opportunities: Successful graduates prioritized for ongoing Expert Contributor roles with high hourly rates
- Requirements: No paid subscriptions required; must be able to upload valid sources (PDF/CSV)
Why Apply?
- Get paid to learn highly technical skills essential for next-phase AI development
- Set the standard for AI models, teaching them the difference between mediocre and expert responses
- Unlock access to Mercor's specialized, high-hourly-rate expert projects
Apply today to join the Rubric Academy and teach AI how to meet your professional standards.
How to apply for this role
- Upload your resume — keep it up-to-date and in English. Mercor will auto-fill your profile from it.
- Complete the AI interview — a 15-minute conversation about your experience. Be ready to discuss specific projects and challenges you've solved.
- Submit your application — only about 20% of applicants finish all the steps, so completing yours puts you well ahead.