Bilingual AI Evaluator (Japanese) at Mercor

Bilingual AI Evaluator (Japanese–English) | $30–40/hr | Worldwide Remote

Mercor is looking for Internet-Native Bilingual Evaluator Experts fluent in Japanese and English to support cutting-edge AI research in multilingual communication, internet culture, and digital language. If you're deeply embedded in online communities across both languages and understand how culture, humour, and slang differ between them, this role is for you.

What You'll Do

• Evaluate Internet-Native Communication: Analyse content from social media, forums, gaming communities, and video platforms. Assess AI-generated outputs for linguistic quality, cultural accuracy, and authenticity — including slang, memes, emoji usage, honorifics, and code-switching.
• Develop Evaluation Standards: Build rubrics for bilingual internet communication, document cultural and linguistic edge cases, and establish guidelines for tone, appropriateness, and contextual understanding.
• Conduct Model Testing & Feedback: Test AI systems using bilingual prompts and real-world internet scenarios. Provide detailed, actionable feedback to improve multilingual model performance.
• Support Quality Assurance: Participate in benchmark development and QA reviews to ensure consistency and quality across evaluation datasets.

Minimum Qualifications

• Native or near-native fluency in Japanese; professional fluency in English
• Deep familiarity with Japanese and English internet culture
• Active presence on platforms such as X, YouTube, Discord, Reddit, Twitch, Nico Nico, or similar communities
• Strong writing, analytical, and critical thinking skills
• Understanding of internet slang, memes, etiquette, and online communication norms in both languages
• Availability to commit 10–20 hours per week

Preferred Qualifications

• Background in linguistics, translation, localisation, journalism, social media, or AI evaluation
• Experience evaluating user-generated content or online communities
• Familiarity with internet subcultures, fandoms, and contemporary digital trends
• Interest in AI, language models, and multilingual communication research