Senior Software Engineer (AI Eval) at Mercor

Senior Software Engineer (AI Evaluation) | $50–75/hr | Remote (US/Canada) | ~1 Week Contract

Join a cutting-edge AI evaluation project and help shape the future of AI-assisted software development. We're seeking experienced Software Engineers to assess AI-generated code through the lens of a seasoned engineer, contributing directly to the improvement of frontier language models.

Project Details:

• Duration: Approximately 1 week (July 8–14)
• Minimum Commitment: 10 hours
• Location Requirement: Must be based in the US or Canada for real-time collaboration

What You'll Do:

• Familiarize yourself with assigned open-source codebases and their engineering practices
• Review code changes, technical discussions, and related software artifacts
• Evaluate AI-generated code for correctness, quality, maintainability, and adherence to best practices
• Provide structured, well-reasoned technical evaluations
• Collaborate with fellow reviewers to ensure consistency and quality

Note: This is an evaluation role — you will not be writing code, but applying your engineering expertise to assess AI-generated solutions.

Required Qualifications:

• 3+ years of professional software engineering experience (or equivalent open-source experience)
• Proficiency in one or more of: JavaScript/TypeScript (Node.js, React), Go (backend, distributed systems, REST/gRPC), or Python (backend, DevOps, testing frameworks)
• Experience working with large production codebases and unfamiliar repositories
• Strong understanding of Git and GitHub workflows (branching, PRs, code reviews)
• Solid grasp of software engineering best practices and design principles
• Strong English communication skills

Nice to Have:

• Experience with Kubernetes or cloud-native development
• Familiarity with open-source development workflows
• Knowledge of SWE-Bench or SWE-Bench Pro