Bilingual German AI Evaluator Expert at Mercor

Bilingual German AI Evaluator | $25–30/hr | Remote in Germany

Mercor is seeking native German speakers with exceptional writing skills to contribute to a high-impact AI research project with a leading lab. This short-term, flexible opportunity combines language mastery, critical thinking, and instructional clarity to train and evaluate advanced language models through culturally grounded German content.

What You'll Do

• Multilingual Prompt Design: Create detailed prompts in German and English with multiple constraints, ensuring natural phrasing and real-world relevance for German-speaking users
• Define Evaluation Standards: Establish high-level expectations for correct responses in German consumer contexts and develop comprehensive rubrics that account for linguistic nuance, tone, and cultural conventions
• Model Testing & Grading: Run prompts through models and assess outputs for accuracy, fluency, and cultural fit in German, comparing results against English where needed
• Quality Assurance: Collaborate in QA review processes to ensure prompt tasks and rubrics maintain consistency and reliability across German-language benchmarks

Minimum Qualifications

• Native-level fluency in German (written) with strong reading/writing ability in English
• BS or BA from a reputable institution (completed or in progress)
• Strong writing and critical thinking skills
• Ability to work independently and meet deadlines
• Significant familiarity with ChatGPT or similar AI tools
• Based in Germany or able to reliably produce Germany-specific, culturally accurate German content

Preferred Qualifications

• Experience in teaching, research, editing, or academic writing
• Experience creating evaluation criteria, rubrics, or grading guidelines
• Familiarity with LLMs, prompting, or model evaluation

Project Details

• Commitment: 20+ hours per week for approximately 2+ months
• Application process: 15-minute AI interview + 45-minute written assessment
• Structured project environment with clear goals and tools

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.