Healthcare Expert (MD, DO, RN, NP, PA, MPH) at Mercor

Healthcare AI Evaluation Expert | $189/hr | Worldwide Remote

Mercor is seeking experienced healthcare professionals to evaluate and improve conversational AI systems used in medical and healthcare contexts. Apply your clinical expertise to ensure AI-generated healthcare responses are accurate, clear, and reliable.

Why This Role Exists

Mercor partners with leading AI teams to enhance the quality and reliability of conversational AI systems. In healthcare scenarios, accuracy is critical. Your expertise will help ensure AI responses to medical queries are factually correct, clearly explained, and aligned with real-world healthcare standards.

What You'll Do

• Write and refine prompts to guide AI model behavior in healthcare scenarios
• Evaluate LLM-generated responses to healthcare queries for accuracy, reasoning, clarity, and completeness
• Conduct fact-checking of medical and healthcare claims using trusted authoritative sources
• Annotate model responses by identifying strengths, improvements, and inaccuracies
• Assess tone, completeness, and appropriateness of responses for real-world healthcare use
• Ensure model responses align with expected conversational behavior and system guidelines
• Apply consistent evaluation standards using clear taxonomies and detailed guidelines

Who You Are

• Minimum 5 years of professional healthcare experience with an expert degree (MD, DO, RN, NP, PA, PharmD, MPH, or equivalent)
• Experience in one or more sub-domains: General Clinical Care, Specialty Medicine/Surgery, Diagnostics/Imaging/Laboratory Medicine, or Public Health/Healthcare Systems
• Significant experience using large language models (LLMs) and understanding their applications
• Excellent written communication skills for complex medical topics
• Strong attention to detail with ability to evaluate clinical reasoning and identify subtle inaccuracies
• Fluent in English

Nice-to-Have Specialties

• Prior experience with RLHF, model evaluation, or data annotation
• Experience writing or editing high-quality medical content
• Clinical documentation or patient communication experience
• Familiarity with evaluation rubrics or quality scoring systems

What Success Looks Like

• Identify medical inaccuracies, unclear explanations, or unsafe reasoning patterns
• Improve clarity and reliability of healthcare-related AI responses through detailed feedback
• Deliver reproducible evaluation artifacts that strengthen model performance
• Build trust in AI systems for healthcare contexts through rigorous evaluation

Why Join Mercor

Play a direct role in shaping how AI systems communicate about medical topics. Apply your healthcare expertise in a unique setting while contributing to the development of more accurate and reliable healthcare AI systems that serve users worldwide.