This job post has expired on September 05, 2025. It is likely that the position has already been filled.
General Knowledge Evaluator (Remote, Multidisciplinary)
Company: Mercor
Hourly Rate: $20–$35/hour (based on experience)
Location: Remote (India)
Positions Available: 20
Application Deadline: August 9, 2025
Mercor invites PhD holders and current doctoral candidates with multidisciplinary academic expertise to participate in an impactful AI research project in collaboration with a premier AI lab. In this role, you'll assess and enhance large language models (LLMs) through rigorous evaluations of their performance on general knowledge and reasoning tasks spanning various academic fields.
Evaluate model-generated content for accuracy, clarity, and logical reasoning across diverse domains such as science, humanities, social sciences, and current affairs.
Conduct structured evaluations using detailed rubrics (8–10 questions), providing clear and concise written feedback.
Identify and document inaccuracies, logical errors, and unclear explanations.
Work independently, asynchronously, and efficiently utilizing provided tools and guidelines.
PhD (or current doctoral candidate) in sciences, humanities, social sciences, or a relevant multidisciplinary area.
Broad-based general knowledge across multiple academic and intellectual disciplines.
Exceptional critical thinking, analytical, and writing abilities.
Meticulous attention to detail and comfort with independent, remote workflows.
Basic Python skills or willingness to learn (preferred but not mandatory).
Fully remote and asynchronous position offering flexible scheduling (10–20 hours/week).
Opportunity to significantly contribute to the advancement of AI research.
Structured workflows with clear guidelines and ongoing support.
Competitive hourly rate between $20–$35, depending on expertise and experience.
Weekly payments securely processed via Stripe Connect.
Engagement as an independent contractor through Mercor.
Mercor, headquartered in San Francisco, connects elite professionals with leading AI research projects. Backed by renowned investors including Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey, Mercor empowers AI labs with specialized human expertise to accelerate groundbreaking advancements.
Mercor promotes diversity and inclusion, welcoming all qualified applicants regardless of protected characteristics and providing accommodations upon request.