AI Code Reviewer (Python) at Turing

AI Code Reviewer (Python) | $20 per approved task | Remote

Turing, which works with frontier AI labs on training data, evaluations and engineering talent, is hiring experienced (tech-lead-level) software engineers to review and rate AI-generated code. The role involves hands-on engineering work, including development environment automation, issue triaging, and evaluating test coverage and quality.

What the work involves

• Analyze and triage GitHub issues across trending open-source libraries.
• Set up and configure code repositories, including Dockerization and environment setup.
• Evaluate unit test coverage and quality.
• Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
• Collaborate with researchers to identify repositories and issues that are challenging for LLMs.

Requirements

• Minimum 2+ years of software engineering experience.
• Strong experience with Python.
• Proficiency with Git, Docker and basic software pipeline setup.
• Ability to understand and navigate complex codebases.
• Comfortable running, modifying and testing real-world projects locally.
• Experience contributing to or evaluating open-source projects is a plus.

Nice to have

• Previous participation in LLM research or evaluation projects.
• Experience building or testing developer tools or automation agents.

Pay, schedule and contract

• Payment: $20 per approved task.
• Minimum 40 hours per week, with a 4-hour overlap with Pacific Time (PST).
• Contractor assignment (no medical/paid leave).
• Contract duration: 2-3 weeks.

Eligible locations

Applicants must be based in India, Pakistan, Nigeria, Egypt, Ghana, Bangladesh, Turkey or Mexico.

Hiring process

An assessment-based review, taking approximately 75 minutes.