Audio Generalist Evaluator Expert at Mercor

Audio Generalist Evaluator Expert (Australian English) | $50/hr | Worldwide Remote

Mercor is hiring an Australian English Audio Generalist Evaluator Expert to support a high-impact AI audio research project with a leading research lab. This short-term, structured engagement involves transcription, annotation, and evaluation tasks that help train and benchmark advanced language models. Ideal for detail-oriented candidates with strong analytical skills and native or near-native fluency in Australian English.

Key Responsibilities

• Transcribe & Optimize Audio/Video: Listen to, analyze, and transcribe audio and video content in Australian English, capturing tone, intent, register, regional expressions, slang, and contemporary usage with precision.
• Define Evaluation Standards: Establish clear quality benchmarks, develop detailed rubrics and grading guidelines, and document standards to ensure consistency across reviewers and model evaluations.
• Conduct Model Testing & Grading: Run prompts through language models, assess outputs against criteria for accuracy, fluency, completeness, and instructional clarity, and provide structured feedback.
• Support Benchmarking & QA: Participate in quality assurance cycles, maintain consistency across datasets, and collaborate with project leads to resolve ambiguities and refine task design.

Minimum Qualifications

• Native or near-native fluency in Australian English (spoken and written)
• Strong writing, editing, and critical thinking skills
• Familiarity with Australian English vocabulary, accents, slang, and contemporary usage
• Ability to accurately transcribe and analyze English audio across general consumer contexts
• Ability to work independently and commit 10–20 hours per week

Preferred Qualifications

• College students or recent graduates welcome
• Background in linguistics, humanities, journalism, translation/localization, or social sciences
• Prior experience with transcription, annotation, localization, or research workflows
• Familiarity with differences between Australian, New Zealand, and other English varieties
• Interest in AI, language models, or applied research environments

Application Process

Complete a short AI-led interview (~15 minutes). Selected candidates will be onboarded and invited to begin project work promptly.