Bilingual Spanish AI Evaluator Expert at Mercor

Bilingual Spanish AI Evaluator | $25–30/hr | Remote (Spain/Mexico)

Mercor is seeking native Spanish speakers with exceptional writing skills to contribute to a high-impact AI research project with a leading lab. This short-term, flexible opportunity is ideal for professionals who combine language mastery, strong critical thinking, and instructional clarity. You'll author Spanish/English prompt–golden answer pairs that train and evaluate advanced language models while maintaining cultural authenticity and technical precision.

Key Responsibilities:

• Multilingual Prompt Design & Optimization: Create detailed prompts in Spanish and/or English with multiple constraints, ensuring natural phrasing and real-world relevance for Spanish-speaking users.
• Define Evaluation Standards: Establish high-level expectations for correct responses in Spanish consumer contexts and develop comprehensive rubrics that account for linguistic nuance, tone, and cultural conventions.
• Model Testing and Grading: Run prompts through AI models and assess outputs for accuracy, fluency, and cultural fit in Spanish, comparing results against English where needed.
• Benchmarking & Quality Assurance: Collaborate in QA review processes to ensure prompt tasks and rubrics meet rigor standards, maintaining consistency across Spanish-language benchmarks.

Minimum Qualifications:

• Native-level fluency in Spanish (written) with strong reading/writing ability in English
• BS or BA from a reputable institution (completed or in progress)
• Strong writing and critical thinking skills
• Ability to work independently and meet deadlines
• Significant familiarity with ChatGPT or similar AI tools
• Based in Spain or Mexico (or able to produce culturally accurate Spanish for these regions)

Preferred Qualifications:

• Experience in teaching, research, editing, or academic writing
• Experience creating evaluation criteria, rubrics, or grading guidelines
• Familiarity with LLMs, prompting, or model evaluation

Project Details:

• Commitment: At least 20 hours per week for approximately 2+ months
• Application process includes a 15-minute AI-led interview and a 45-minute written assessment
• Structured project environment with clear goals and tools

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.