Professional Domain Expert (IT) at Mercor

IT Domain Expert (AI Evaluator) | $40–60/hr | Worldwide Remote

Mercor is partnering with a leading AI lab to bring on experienced Information Technology professionals for an AI model evaluation project. As a contractor, you'll play a direct role in improving the accuracy, safety, and reliability of AI-generated responses in high-stakes IT contexts — where misinformation can have serious consequences.

This is a flexible, part-time engagement (~20 hrs/week) ideal for seasoned IT practitioners or advisors looking to contribute to cutting-edge AI development.

Key Responsibilities

• Craft realistic prompts that reflect how IT professionals and consumers seek domain-specific guidance
• Evaluate AI-generated responses for factual accuracy, technical correctness, and practical usefulness
• Identify fabricated claims, incorrect references, or flawed reasoning in model outputs
• Score and rank multiple AI responses using structured rubrics across key quality dimensions
• Provide detailed written justifications supported by specific evidence for each evaluation

Ideal Qualifications

• Master's degree or higher in Computer Science, Information Systems, or a related field
• Hands-on professional experience in an IT practitioner or advisory role
• Familiarity with industry standards, compliance frameworks, or IT regulations
• Strong written communication and critical analytical skills

Application Process

1. Submit your resume to get started
2. Complete the Model Response Evaluation assessment

This is a contractor role with an expected commitment of approximately 20 hours per week. Join a project that directly shapes the future of trustworthy AI.