QA Specialist – Audio Annotation & Diarization (US English) | Contractor | Remote (US)
Turing is seeking a detail-oriented QA Specialist to serve as the final quality checkpoint for an evaluation-grade, multi-channel audio transcription and diarization dataset. This role is ideal for linguists, professional transcriptionists, or language educators with a sharp ear for audio fidelity and a meticulous approach to verbatim transcription accuracy.
About Turing
Based in San Francisco, CA, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing helps customers accelerate frontier research with high-quality data, advanced training pipelines, and top AI researchers — and helps enterprises transform AI from proof of concept into proprietary, measurable intelligence.
Role Overview
This project involves building a highly accurate dataset of transcribed, multi-channel audio recordings used to evaluate multilingual, multi-speaker AI systems. Contributors record unscripted group conversations, which are transcribed and diarized, then human-validated. As a QA Specialist, you will review the end-to-end quality of both the audio recordings and the human-verified annotations.
Key Responsibilities
Audio Quality Assurance
- Evaluate multi-channel audio recordings against strict technical and fidelity standards.
- Verify channel isolation (no audio bleed) and confirm recordings were captured in quiet, appropriate environments free from clipping, low gain, or disruptive background noise.
Transcription & Diarization Verification
- Review human-validated transcriptions to ensure exceptionally high accuracy and adherence to low Word Error Rate (WER) targets.
- Confirm transcripts accurately capture spontaneous, unnormalized speech — including overlaps, interruptions, and false starts.
- Validate turn-level and word-level timestamps and speaker identification, with particular attention to complex overlapping dialogue.
- Read and validate underlying JSON-formatted data to ensure accurate metadata tagging and timestamp logic.
Metadata & Content Review
- Verify accuracy of all applied metadata, including demographic markers, contextual domains, and conversational tags.
- Audit sessions to enforce strict safety and privacy standards — ensuring no PII, toxic, or sensitive content is present.
Execution & Reporting
- Assign clear pass/fail or agree/disagree statuses during review.
- Provide detailed, actionable feedback whenever disagreeing with an annotator's work.
Requirements & Qualifications
- Native proficiency in US English.
- Exceptional ear for audio fidelity — able to detect subtle background noise, channel bleed, or clipping.
- Meticulous attention to detail for word-level timestamps and strict verbatim transcription.
- Ability to accurately assess complex multi-speaker conversational dynamics.
Ideal backgrounds include:
- Linguists / Phonetics Experts: Deep understanding of natural speech patterns; expertise in annotating overlaps, false starts, and backchannels.
- Language Teachers: Native-level mastery with ability to adhere strictly to verbatim transcription guidelines, documenting every disfluency without prescriptive corrections.
- Professional Transcriptionists: Strong audio fidelity instincts, rigorous timestamping skills, and experience meeting strict accuracy targets.
Engagement Details
- Type: Contractor / Freelancer (no medical or paid leave benefits)
- Duration: 15 weeks
- Commitment: Minimum 4 hours/day, 40 hours/week, with at least 4 hours of overlap with PST
- Environment: Fully remote
Evaluation Process
Shortlisted candidates will be reviewed internally by the Turing team and contacted directly for onboarding.
