C# AI Model Evaluator at Turing

C# AI Model Evaluator | $25/task | Remote (Asia, Africa, LATAM)

Turing is hiring experienced software engineers for an AI training and evaluation initiative focused on improving the coding capabilities of AI agents. This is a 1-month, fully remote contractor assignment.

What the work involves

For each task, three AI coding agents are given the same real developer request. You review each agent's full run — the commands it ran, the files it changed, what it checked and its final message — then judge which run did the job best. For every task you:

• Rate each run on 10 quality areas, such as following instructions, whether the code works, efficiency, safety, honesty and communication.
• Give each run an overall score from 1 to 5 and rank the three.
• Write a short rationale explaining your ranking, with evidence from the runs.

Requirements

• Minimum 3+ years of overall software engineering experience.
• Strong experience with C#.
• Proficiency with Shell commands, Git, Docker and basic software pipeline setup.
• Comfortable running, modifying and testing real-world projects locally.

Assessment and selection

• Candidates must complete three practice evaluation tasks; each requires quality ratings, overall scores, comparative rankings and written rationales.
• A reviewer assesses each task for technical accuracy, consistency and quality of reasoning.
• Onboarding happens only after all three practice tasks have been approved.

Strict no-AI policy: Do not use ChatGPT or any other AI tool to write your evaluation rationales or reasoning. All rationales must be written independently by the candidate. Compliance will be checked, and any use of AI-generated rationales results in automatic rejection.

Pay, schedule and contract

• Payment: $25 per approved task.
• Commitment: at least 8 hours per day and 40 hours per week, with 4 hours of overlap with PST.
• Duration: 1-month contractor assignment (no medical or paid leave); expected start date is next week.

Location

Fully remote. Open to applicants in India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico and Latin America.