Model Evaluator (Go) at Turing

Model Evaluator (Go) | $25/task | Remote — eligible countries only

Contract role with Turing, an AI company, supporting a frontier AI training and evaluation initiative focused on improving the coding capabilities of AI agents.

What the work involves

You review and compare AI-generated coding solutions across real-world software engineering tasks. For each task, three AI coding agents are given the same real developer request. You read each agent's full run — the commands it ran, the files it changed, what it checked and its final message — and judge which run did the job best.

• Rate each run on 10 quality areas, such as following instructions, whether the code works, efficiency, safety, honesty and communication.
• Give each run an overall score from 1 to 5 and rank the three.
• Write a short rationale explaining your ranking, with evidence from the runs.

Requirements

• Minimum 3+ years of software engineering experience.
• Strong experience with Go.
• Proficiency with shell commands, Git, Docker and basic software pipeline setup.
• Comfortable running, modifying and testing real-world projects locally.

Pay and schedule

• $25 per approved task.
• Contractor assignment (no medical or paid leave); 1-month contract, expected start next week.
• At least 8 hours per day and 40 hours per week, with 4 hours of overlap with PST.
• Fully remote.

Where applicants can be located

India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico, LA.

Selection process

• Complete three practice evaluation tasks as part of selection. Each requires quality ratings, overall scores, comparative rankings and written rationales, and is reviewed for technical accuracy, consistency and quality of reasoning.
• You are onboarded only after all three practice tasks are approved.
• Evaluation process: approximately 75 minutes, assessment-based review.
• Strict no-AI policy: do not use ChatGPT or any other AI tool to write your rationales, explanations or reasons. All rationales and reasoning must be written independently by you. Compliance will be checked, and any use of AI-generated rationales results in automatic rejection.