Personal Account Generalist Rater | Contractor | Remote (US Only)
Turing is seeking US-based generalist raters to evaluate personalized AI interactions for Google's Gemini platform. This short-term contract role involves assessing AI-generated responses using your connected Google apps to help improve personalization quality and model performance.
About Turing
Based in San Francisco, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing specializes in high-quality data, advanced training pipelines, and AI expertise across coding, reasoning, STEM, multilinguality, and more.
Role Overview
As a Generalist Rater, you will participate in Gemini Personalization Evals, performing evaluation tasks using personalized prompts that retrieve context and data across connected Google apps such as Gmail, Google Photos, Google Calendar, and Google Drive.
Key Responsibilities
- Evaluate AI-generated responses based on your personal email history and account activity.
- Assess whether responses are relevant, accurate, helpful, and appropriately personalized.
- Identify issues related to incorrect personalization, unsupported assumptions, or irrelevant recommendations.
- Compare multiple AI responses and determine which provides the better overall user experience.
- Provide clear, detailed, and well-reasoned evaluation feedback to support model improvements.
- Follow project guidelines to ensure consistent, accurate, and high-quality evaluations.
- Maintain data privacy and adhere to all project confidentiality requirements.
Qualifications
- Must be currently based in the United States.
- Willing and able to connect Google apps (Gmail, Google Photos, Google Calendar, Google Drive, etc.) to Gemini for evaluation purposes.
- Active use of standard Google workspace or consumer apps with sufficient personal data history.
- Strong analytical thinking with the ability to evaluate nuanced AI responses.
- Excellent attention to detail and written communication skills.
- Self-motivated and able to work independently in a remote environment.
- Desktop or laptop with a reliable internet connection.
- Bachelor's degree or equivalent practical experience in any field.
- Prior experience in AI evaluation, data annotation, content review, or quality assurance is a plus but not required.
Engagement Details
- Type: Contractor
- Commitment: At least 4 hours/day, up to 40 hours/week, with 4 hours of overlap with PST
- Duration: Up to 4 months
Evaluation Process
- Shortlisted candidates will receive a Job Interest Form.
- Final selected candidates will be contacted with next steps, including consent and onboarding requirements.
Candidates must be willing to grant necessary app integration permissions and complete all required consent forms and non-disclosure agreements.
