Live MLB Game AI Evaluator at Mercor
posted 1 hour agoLive MLB Game AI Evaluator | $100-120/hr | Remote (US) | Task-Based
US-based baseball fans are needed to evaluate AI assistants live during real MLB postseason games. On scheduled game days, you'll ask AI assistants the questions you'd naturally ask while following a game, then rate how helpful, accurate, and well-presented their answers are. Your judgments directly shape how AI products serve sports fans.
What You'll Do
- Follow live MLB postseason games and ask AI assistants real fan questions: live scores and win probability, player and team stats, rules and challenges, schedules and broadcasts, highlights, and expert-style analysis
- Run the same question on two AI products side by side, either as a single question or as a multi-turn conversation of 2 to 10 turns
- Capture share links and full screenshots of every conversation
- Rate each response for helpfulness, factual accuracy, and presentation, choose which product did better, and write a clear rationale for your choice
Requirements
- Based in the United States
- At least 1 year of professional work experience
- Solid baseball knowledge: rules, teams, players, and how the postseason works
- Being an avid MLB fan or a general sports enthusiast is a plus
- Available to work live during game windows on scheduled game days
- Comfortable using AI chat assistants and web search
- Strong written English, sharp attention to detail, and the ability to spot factual errors
- Prior AI rating or evaluation experience is a plus, but not required
Pay and Schedule
- $100-120/hr, task-based
- Work happens on MLB postseason game days in October, from the Division Series through the World Series
- Each task takes about 2 to 2.5 hours and is completed while the game is live
- The first game day is as early as Saturday, October 3
Location
Fully remote, but you must be based in the United States since the work follows live MLB game windows.
All qualified applicants are considered without regard to legally protected characteristics, and reasonable accommodations are provided upon request.
How to apply for this role
- Upload your resume — keep it up-to-date and in English. Mercor will auto-fill your profile from it.
- Complete the AI interview — a 15-minute conversation about your experience. Be ready to discuss specific projects and challenges you've solved.
- Submit your application — only about 20% of applicants finish all the steps, so completing yours puts you well ahead.