LLM Evaluation Domain Expert at Turing

Domain Expert, LLM Evaluation | Pay: Varies | Remote

About the Role

Turing, based in San Francisco, works with frontier AI labs and enterprises on AI research and deployment. The company is hiring Domain Experts (SMEs) to support the evaluation, testing and improvement of advanced Large Language Models (LLMs). You will apply your subject-matter expertise to assess AI-generated responses, identify knowledge gaps, design challenging prompts, and improve model accuracy, reliability and reasoning.

What the Work Involves

• Create complex, domain-specific prompts to test LLM capabilities.
• Evaluate AI-generated responses for factual accuracy, reasoning, relevance and overall quality.
• Identify factual errors, logical inconsistencies, knowledge gaps and model weaknesses.
• Develop challenging test cases and edge cases that expose model limitations.
• Provide clear, detailed, evidence-based feedback to improve model performance.

Minimum Qualifications

• Master's degree or higher in a relevant field.
• At least three years of professional, research or teaching experience.
• Excellent written English communication skills.
• Strong analytical and critical-thinking abilities.
• Exceptional attention to detail.

Preferred Qualifications

• Experience working with LLMs or Generative AI.
• Familiarity with prompt engineering, AI evaluation or model testing.
• Experience designing challenging prompts, identifying edge cases or stress-testing AI models.

Ideal Candidate

Curious, detail-oriented and deeply knowledgeable in your field. You enjoy solving complex problems, challenging assumptions, and identifying subtle factual or reasoning errors.

Pay, Schedule and Location

The listing does not specify pay, working hours, commitment or location restrictions.