Posted over 30 days ago. Roles like this often stay open for a while, so it's still worth applying.
AI Rubric Writer at micro1
posted 3 months agoAI Rubric Writer | $95–185/hr | Worldwide Remote | Contractor
Join MICRO1 as an AI Rubric Writer and play a pivotal role in shaping the quality and reliability of AI systems. In this role, you will design evaluation frameworks, develop scoring criteria, and assess AI-generated outputs to ensure high standards across large language model (LLM) pipelines.
Key Responsibilities
- Develop clear, structured rubrics and scoring criteria for evaluating AI and LLM outputs
- Assess and evaluate prompts, responses, and model behavior for quality and accuracy
- Collaborate with AI training teams to refine evaluation methodologies
- Serve as a subject matter expert in your domain to ensure contextually accurate assessments
- Identify gaps in existing evaluation frameworks and propose improvements
- Document findings and maintain consistent quality standards across projects
Required Skills & Experience
- Experience as a Rubric Writer, AI Evaluator, or Assessment Specialist
- Familiarity with LLM evaluation, prompt engineering, or AI quality assurance
- Strong analytical skills with attention to detail in scoring and assessment
- Ability to create structured, objective evaluation frameworks
- Background as a Subject Matter Expert in AI, machine learning, or a related field is a plus
Compensation
$95–$185/hr, depending on experience and expertise.
This is a fully remote contractor opportunity open to candidates worldwide.