QA / Test Engineer at Mercor

QA / Test Engineer | $60–90/hr | Remote (US Only)

Join a leading AI lab's GenAI team and play a critical role in shaping the quality and integrity of next-generation agentic evaluation benchmarks. As a QA/Test Engineer, you'll be the quality backbone of a frontier AI benchmark program — ensuring every task is airtight, unambiguous, and correctly graded.

This is a full-time W-2 position with Cincinnatus LLC, embedded within a leading AI lab's extended workforce. The role is fully remote within the United States at approximately 35 hours per week.

Key Responsibilities

• Design Test Cases: Build thorough checks that confirm each benchmark task works as intended, including edge cases and corner scenarios.
• Review Tasks: Carefully evaluate tasks and reference solutions before finalization, catching ambiguity and gaps early in the process.
• Debug: Dive into Python code when tasks or grading checks behave unexpectedly, resolving issues end-to-end.
• Shape Quality Processes: Help develop repeatable quality checklists and deliver actionable feedback to task authors.
• Protect Benchmark Integrity: Identify shortcuts and grading gaps in AI agent runs to ensure benchmark scores remain trustworthy and meaningful.

Core Qualifications

• MSc or PhD in a STEM field, or equivalent experience in a research-heavy or engineering-heavy domain.
• 1+ years of experience in test engineering, QA, or a software/research engineering role with strong quality ownership.
• Demonstrated ability to design test cases, build quality-review processes, and debug complex systems.
• Working proficiency in Python and Git, with comfort navigating unfamiliar codebases.
• Exceptional attention to detail and strong written documentation habits.
• Experience in AI training, model evaluation, or quality review of AI-generated content is a strong plus.
• A perfectionist mindset with the creativity to find what others miss, and the ability to work independently through open-ended problems.

About the Employer

Cincinnatus LLC is an enterprise staffing company that partners with leading technology firms to place highly skilled professionals in structured, role-based engagements. Cincinnatus serves as the employer of record, providing W-2 employment, payroll, benefits, and compliance support while integrating employees directly into client teams.

Cincinnatus LLC is an Equal Employment Opportunity employer committed to fostering an inclusive and diverse workplace.