Russian Audio Evaluator Expert at Mercor

Russian Audio Evaluator Expert | $50/hr | San Francisco Bay Area (Part-Time, Contract)

Mercor is seeking a skilled Russian Audio Generalist Evaluator Expert to support a high-impact audio AI research project with a leading research lab. This structured, short-term engagement involves transcription, annotation, and evaluation tasks that help train and benchmark advanced language models. Ideal for fluent Russian speakers with strong analytical backgrounds who are based in the San Francisco Bay Area.

Key Responsibilities

• Transcribe & Analyze Audio/Video: Listen to, analyze, and transcribe Russian-language audio and video content with strict adherence to formatting, stylistic, and accuracy guidelines. Capture nuances including tone, register, dialects, and contemporary Russian usage.
• Define Evaluation Standards: Develop clear rubrics and grading guidelines in both Russian and English. Document standards to ensure consistency across reviewers and model evaluations, identifying linguistic edge cases specific to Russian.
• Model Testing & Grading: Run prompts through language models, assess generated outputs for accuracy, fluency, and completeness, and provide structured feedback to improve model performance.
• Benchmarking & QA Support: Participate in quality assurance cycles, maintain dataset consistency, and collaborate with project leads to refine task design and resolve ambiguities.

Minimum Qualifications

• Native or near-native fluency in Russian (spoken and written) and professional fluency in English
• Strong familiarity with spoken Russian, regional dialects, and contemporary language usage
• Excellent writing, editing, and critical thinking skills
• Ability to work independently and meet deadlines
• Must be based in the San Francisco Bay Area
• Available to commit 10–20 hours per week

Preferred Qualifications

• College students or recent graduates welcome
• Background in linguistics, humanities, journalism, translation/localization, or related fields
• Prior experience with transcription, annotation, or research workflows in Russian
• Interest in AI, language models, or applied research environments

Application Process

Complete a short AI-led interview (approximately 15 minutes). Selected candidates will be onboarded and invited to begin project work promptly.