Bilingual AI Evaluator (Spanish–English) at Mercor

Bilingual AI Evaluator (Spanish–Spain) | $30–40/hr | Remote – Spain

Mercor is looking for an Internet-Native Bilingual Evaluator Expert fluent in Spanish (Spain) and English to support cutting-edge AI research. You'll evaluate and improve AI model outputs with a focus on multilingual communication, internet culture, and real-world language use — specifically within Spain's online communities. This is a flexible part-time role requiring 10–20 hours per week.

Key Responsibilities

• Evaluate Internet-Native Communication: Analyze content from social media, forums, messaging platforms, and digital communities. Assess AI-generated responses for fluency, cultural authenticity, and naturalness — including slang, memes, code-switching, and evolving online language specific to Spain.
• Develop Evaluation Standards: Create rubrics and guidelines for bilingual, internet-native communication. Define standards for tone, cultural relevance, and linguistic accuracy, and document distinctions between Peninsular and Latin American Spanish internet usage.
• Conduct Model Testing & Feedback: Test language models using real-world multilingual prompts in both English and Spanish, then provide structured feedback to improve model performance and cultural understanding.
• Support Quality Assurance: Participate in review cycles, benchmark development, and collaborate with project leads to maintain consistency across datasets and evaluation frameworks.

Minimum Qualifications

• Native or near-native fluency in Spanish (Spain) and professional fluency in English
• Deep familiarity with Spain-based internet culture and online communities
• Active engagement with platforms such as YouTube, TikTok, Reddit, Discord, X, Instagram, Twitch, Forocoches, or similar
• Strong understanding of Peninsular Spanish slang, memes, internet humor, and contemporary digital culture
• Strong writing, analytical, and critical thinking skills
• Availability to commit 10–20 hours per week

Preferred Qualifications

• Background in content moderation, localization, translation, linguistics, social media, journalism, or AI evaluation
• Experience evaluating user-generated content or digital communications
• Familiarity with differences between Spain Spanish and Latin American Spanish online discourse
• Genuine interest in AI, language models, and internet culture research