Bilingual AI Evaluator (Spanish) at Mercor

Bilingual AI Evaluator (Spanish–English) | $25/hr | Worldwide Remote

Mercor is looking for an Internet-Native Bilingual Evaluator Expert fluent in Latin American Spanish and English to support cutting-edge AI research. In this role, you'll evaluate and annotate AI model outputs, helping improve how language models understand modern online communication, internet culture, slang, and multilingual interactions. This is a flexible part-time role requiring 10–20 hours per week.

Key Responsibilities

• Evaluate Internet-Native Communication: Analyze content from social media, forums, messaging platforms, and online communities. Assess AI-generated responses for fluency, cultural appropriateness, and internet authenticity, including slang, memes, code-switching, and evolving digital language specific to Latin America.
• Develop Evaluation Standards: Create rubrics and guidelines for bilingual and internet-native communication. Define standards for tone, cultural relevance, and linguistic accuracy, and document edge cases involving humor, sarcasm, and platform-specific language across Latin American Spanish-speaking countries.
• Conduct Model Testing and Feedback: Test language models using real-world multilingual prompts in both English and Spanish, and provide structured feedback to improve model performance and cultural understanding.
• Support Quality Assurance: Participate in review cycles, benchmark development, and collaborate with project leads to maintain consistency across datasets and evaluation frameworks.

Minimum Qualifications

• Native or near-native fluency in Latin American Spanish with professional English proficiency
• Deep familiarity with internet culture and online communities across one or more Latin American Spanish-speaking countries
• Active engagement with platforms such as YouTube, TikTok, Reddit, Discord, X, Instagram, Twitch, or WhatsApp communities
• Strong understanding of Latin American Spanish slang, memes, internet humor, and contemporary digital culture
• Strong writing, analytical, and critical thinking skills
• Availability to commit 10–20 hours per week

Preferred Qualifications

• Background in content moderation, localization, translation, linguistics, social media, journalism, or AI evaluation
• Familiarity with regional linguistic differences across Mexico, Argentina, Colombia, Chile, Peru, and other Latin American countries
• Experience evaluating user-generated content or digital communications
• Interest in AI, language models, and internet culture research