About Turing:
Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.
Role Overview
We are seeking highly detail-oriented and analytical AI Data Trainers to join a pivotal project focused on the end-to-end (E2E) evaluation of Gemini’s new personalization capabilities. This initiative is critical to the future of generative AI, aiming to refine how models leverage private data sources—such as Gmail, Google Photos, Google Search, YouTube, and Google Health—to deliver contextually relevant, timely, and safe user experiences.
As an AI Data Trainer, you will act as a human-in-the-loop evaluator. Your role is to bridge the gap between algorithmic outputs and human intent. You will be responsible for testing the personalization features across diverse, real-world user scenarios. Your feedback and ratings will directly influence the fine-tuning process, helping developers understand where the model excels at connecting user context and where it requires further grounding to avoid hallucinations or irrelevant suggestions.
Key Responsibilities
- Advanced Model Interaction: Conduct free-form, multi-turn conversations with Gemini models. You will perform comparative analyses (Head-to-Head testing) between model variants to identify improvements in response quality, tone, and contextual awareness.
- Rigorous Quality Assessment: Evaluate model outputs based on a multi-dimensional rubric, including:
- Grounding: Assessing whether the model correctly retrieves and utilizes personal data (e.g., email content, photos) without fabricating information.
- Personalization Utility: Measuring how well the model anticipates the user’s needs based on their personal data history.
- Safety & Accuracy: Identifying potential privacy leaks or inaccurate interpretations of personal health or sensitive information.
- Structured Use Case Evaluation: Execute predefined "Health & Wellness" scenarios. You will simulate complex user intent to test the model's ability to navigate integrated data ecosystems, such as syncing health data from wearables with queries regarding physical activity or nutrition.
- Deep-Dive Reporting: Beyond simple rating, you will provide qualitative feedback and technical debug logs. This includes documenting specific "reasoning paths" the AI took, identifying failure points in the RAG (Retrieval-Augmented Generation) pipeline, and articulating why one response is superior to another.
Requirements & Eligibility
Age & Eligibility
- Must be 18+ years old and authorized to participate in data training projects.
- Regional Availability
- Must be located in and/or operate using en-US or en-IN language settings to ensure cultural and linguistic nuance.
- Google Account Standards
- Strict Requirement: You must use your primary, daily-use personal Google account. We cannot accept testing or "dummy" accounts, as the model requires a rich, historical data footprint (Gmail, Photos, etc.) to perform personalization evaluations.
- Hardware Ecosystem
- Must possess and actively use a Fitbit device or Pixel Watch. This hardware is a mandatory prerequisite for the "Health & Wellness" integration testing.
Offer Details:
- Commitments Required: at least 4 hours per day and upto 40 hours per week with 4 hours of overlap with PST.
- Engagement type: Contractor
- Engagement Length: 10 weeks
Evaluation Process -
- Shortlisted candidates will be sent a Job Interest Form.
- After the profile review, an assessment will be shared, which must be completed within 24 hours.
- Based on the assessment outcomes, shortlisted candidates will be contacted to discuss the pre‑onboarding requirements.
