AI Safety Annotation Reviewer at Turing

AI Safety Annotation Reviewer | Pay not stated | Remote | Full-time hours (40/week)

Turing is hiring an AI Safety Annotation Reviewer to work with a US-based AI research company. The role involves auditing and validating human-generated annotations against an AI safety policy framework.

What the work involves

• Audit trainer-submitted evaluations for policy misapplication, incomplete reasoning, scoring drift, schema errors, and annotation inconsistencies.
• Apply a detailed AI safety policy framework consistently across high-stakes review tasks.
• Identify individual and systematic errors in human-generated annotations and written assessments.
• Validate annotation completeness, scoring accuracy, policy alignment, and adherence to required schemas.
• Give direct, constructive feedback to trainers and recommend corrective actions when quality issues are found.
• Document recurring error patterns and contribute to quality improvement, calibration, and reviewer training initiatives.
• Maintain accuracy and consistency under time-sensitive conditions and while handling sensitive content.

Requirements

• Sharp attention to detail with strong analytical and logical reasoning skills.
• Demonstrated experience identifying systematic errors in human-generated annotations, evaluations, or written assessments.
• Ability to quickly understand and consistently apply detailed policy guidelines.
• Native-level English fluency and strong written communication skills.
• Comfort providing clear, direct, and constructive feedback.
• Strong understanding of US cultural context and communication norms.
• Ability to work independently, manage priorities, and maintain quality under time pressure.
• Availability to commit to 40 hours per week; immediate availability is preferred.

Preferred qualifications

• Experience in AI annotation review, Trust & Safety quality assurance, test engineering, content moderation, or editorial fact-checking.
• Previous experience with schema validation, policy compliance reviews, annotation completeness checks, or quality auditing.
• Experience working with sensitive or high-stakes content.

Content warning

This project includes sensitive content involving topics such as self-harm, violence, and abuse. Appropriate waivers, support resources, and wellbeing measures will apply.

Pay, schedule and location

• Pay is not stated in the listing; the company describes compensation as competitive.
• Full-time availability of 40 hours per week is required; immediate availability is preferred.
• Flexible working hours in a remote work environment.

Application process

Shortlisted candidates will be contacted.