Software Engineering Expert at Mercor

Software Engineering Expert | $60–90/hr | Remote (US Only)

Join a leading AI lab's GenAI team as a Software Engineering Expert, where you'll design and evaluate cutting-edge agentic benchmarks for frontier AI models. This is a full-time W-2 position (approx. 35 hrs/week) with Cincinnatus LLC, fully remote within the United States.

About the Role

A top-tier AI research lab is building the next generation of agentic evaluation benchmarks and needs experienced software engineers to serve as task curators. You'll design, implement, and review complex, multi-step engineering challenges that simulate real-world problems — tasks that today's best AI coding agents cannot yet reliably solve. Each task represents one to two days of focused effort and spans Python implementation, environment setup, debugging, and documentation.

Key Responsibilities

• Design Tasks: Create realistic, multi-step software engineering challenges that push the limits of state-of-the-art AI coding agents.
• Build Reference Solutions: Solve your own tasks in Python, complete with setup scripts and verification checks for clear, reproducible answers.
• Work with AI Tools: Integrate AI coding assistants into your daily workflow and document where they succeed or fall short.
• Review & Refine: Evaluate tasks created by fellow experts, providing feedback on clarity, correctness, and difficulty calibration.
• Analyze Failures: Study how AI agents attempted your tasks and help researchers understand failure modes.

Core Qualifications

• MSc or PhD in Computer Science or a STEM field, or equivalent practical experience in a research-heavy, coding-intensive domain.
• 1+ years of experience in a research, research-engineering, or software engineering role.
• Strong hands-on Python scripting and debugging skills with clean-code habits.
• Fluency with Git, IDEs, and standard software development workflows.
• Preferred: Experience with AI coding assistants, prompt engineering, or agent workflows.
• Preferred: Background in AI training, model evaluation, or benchmark authoring.
• High attention to detail, creativity, strong written communication, and ability to work independently on open-ended problems.

Employment Details

This role is a structured, full-time W-2 position administered by Cincinnatus LLC, an enterprise staffing company that partners with leading technology firms. Cincinnatus serves as the employer of record, providing payroll, benefits, and compliance support while placing you directly within the client's team.

Cincinnatus is an Equal Employment Opportunity employer and is committed to providing reasonable accommodations for qualified individuals with disabilities.