Personal Account Generalist Rater (Korean) | Contractor | Remote in US
Turing is seeking US-based Korean-speaking generalist raters to evaluate personalized AI interactions for Google's Gemini platform. This short-term contract role involves assessing AI-generated responses across connected Google apps to help improve personalization quality and model performance.
About Turing
Based in San Francisco, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing accelerates frontier research with high-quality data, advanced training pipelines, and top AI researchers specializing in coding, reasoning, STEM, multilinguality, multimodality, and agents.
Role Overview
As a Generalist Rater, you will participate in Gemini Personalization Evals — evaluating AI responses generated from personalized prompts that retrieve context and data across connected Google apps such as Gmail, Google Photos, Google Calendar, and Google Drive.
Key Responsibilities
- Evaluate AI-generated responses based on your personal email history and account activity.
- Assess whether responses are relevant, accurate, helpful, and appropriately personalized.
- Identify issues related to incorrect personalization, unsupported assumptions, or irrelevant recommendations.
- Compare multiple AI responses and determine which provides the better overall user experience.
- Provide clear, detailed, and well-reasoned evaluation feedback to support model improvements.
- Follow project guidelines to ensure consistent, accurate, and high-quality evaluations.
- Maintain data privacy and adhere to all project confidentiality requirements.
Key Qualifications
- Location: Must be currently based in the United States.
- App Connectivity: Must be willing to connect Google apps (Gmail, Google Photos, Google Calendar, Google Drive, etc.) to Gemini for data retrieval and evaluation purposes.
- Account Activity: Active use of standard Google workspace or consumer apps with sufficient personal data and history.
- Analytical Thinking: Ability to evaluate nuanced AI responses and identify strengths and weaknesses in personalization quality.
- Attention to Detail: Strong ability to identify subtle issues or inconsistencies in AI-generated responses.
- Communication: Excellent written communication skills for structured evaluation feedback.
- Independence: Self-motivated and able to work remotely with minimal supervision.
- Technical Setup: Desktop or laptop with a reliable internet connection.
Education & Experience
- Bachelor's degree or equivalent practical experience in any field.
- Experience in AI evaluation, data annotation, content review, or quality assurance is preferred but not required.
Engagement Details
- Type: Contractor
- Commitment: At least 4 hours per day, up to 40 hours per week, with 4 hours of overlap with PST.
- Duration: Up to 16 weeks
Evaluation Process
- Shortlisted candidates will receive a Job Interest Form.
- Final selected candidates will be contacted with next steps, including consent and onboarding requirements.
Consent & Compliance
Candidates must be willing to grant necessary app integration permissions and complete all standard consent forms and non-disclosure agreements prior to onboarding.
