This job post has expired on April 26, 2026. It is likely that the position has already been filled.
iOS/macOS AI Evaluator Expert at Mercor
posted 6 months agoiOS/macOS AI Evaluator Expert | $30–35/hr | Remote (US & Canada)
Mercor is partnering with a leading AI research lab to find detail-oriented professionals who can evaluate and benchmark advanced language models. This is a flexible, short-term freelance opportunity ideal for individuals with strong academic backgrounds, a passion for clear writing, and hands-on experience with Apple platforms.
What You'll Do
- Design & Optimize Prompts: Craft detailed, multi-constraint prompts that effectively test AI model capabilities.
- Conduct Model Testing & Grading: Run prompts through AI models, assess outputs against expectations, and provide clear, reasoned evaluations.
- Support Quality Assurance: Collaborate in QA review processes to ensure prompt tasks and rubrics meet rigorous standards before integration into official benchmarks.
Minimum Qualifications
- BS or BA from a reputable institution (completed or in progress)
- Strong writing and critical thinking skills
- Ability to work independently and meet deadlines
- Familiarity with ChatGPT or similar AI tools
- Daily use of macOS and/or iOS
- Based in the US or Canada
Preferred Qualifications
- Background in teaching or academic research
Commitment & Schedule
- Minimum 20 hours per week
- Initial commitment of 2–3 weeks, with strong potential to extend
- Structured project environment with clear goals and tools provided
Application Process
Complete a brief AI-led interview (approx. 15 minutes). Qualified candidates will be invited to join the project shortly after.
How to apply for this role
- Upload your resume — keep it up-to-date and in English. Mercor will auto-fill your profile from it.
- Complete the AI interview — a 15-minute conversation about your experience. Be ready to discuss specific projects and challenges you've solved.
- Submit your application — only about 20% of applicants finish all the steps, so completing yours puts you well ahead.