This job post has expired on July 18, 2026. It is likely that the position has already been filled.
Audio Generalist Evaluator Expert at Mercor
posted 3 months agoAudio Generalist Evaluator Expert (Australian English) | $50/hr | Worldwide Remote
Mercor is hiring an Australian English Audio Generalist Evaluator Expert to support a high-impact AI audio research project with a leading research lab. This short-term, structured engagement involves transcription, annotation, and evaluation tasks that help train and benchmark advanced language models. Ideal for detail-oriented candidates with strong analytical skills and native or near-native fluency in Australian English.
Key Responsibilities
- Transcribe & Optimize Audio/Video: Listen to, analyze, and transcribe audio and video content in Australian English, capturing tone, intent, register, regional expressions, slang, and contemporary usage with precision.
- Define Evaluation Standards: Establish clear quality benchmarks, develop detailed rubrics and grading guidelines, and document standards to ensure consistency across reviewers and model evaluations.
- Conduct Model Testing & Grading: Run prompts through language models, assess outputs against criteria for accuracy, fluency, completeness, and instructional clarity, and provide structured feedback.
- Support Benchmarking & QA: Participate in quality assurance cycles, maintain consistency across datasets, and collaborate with project leads to resolve ambiguities and refine task design.
Minimum Qualifications
- Native or near-native fluency in Australian English (spoken and written)
- Strong writing, editing, and critical thinking skills
- Familiarity with Australian English vocabulary, accents, slang, and contemporary usage
- Ability to accurately transcribe and analyze English audio across general consumer contexts
- Ability to work independently and commit 10–20 hours per week
Preferred Qualifications
- College students or recent graduates welcome
- Background in linguistics, humanities, journalism, translation/localization, or social sciences
- Prior experience with transcription, annotation, localization, or research workflows
- Familiarity with differences between Australian, New Zealand, and other English varieties
- Interest in AI, language models, or applied research environments
Application Process
Complete a short AI-led interview (~15 minutes). Selected candidates will be onboarded and invited to begin project work promptly.
How to apply for this role
- Upload your resume — keep it up-to-date and in English. Mercor will auto-fill your profile from it.
- Complete the AI interview — a 15-minute conversation about your experience. Be ready to discuss specific projects and challenges you've solved.
- Submit your application — only about 20% of applicants finish all the steps, so completing yours puts you well ahead.