SWE Bench Engineer - C++ at Turing

SWE Bench Engineer - C++ | $25/task | Remote contractor, 2-3 weeks

Turing, an AI company, is hiring experienced C++ engineers to evaluate how large language models (LLMs) handle real code. The project combines hands-on software engineering with AI evaluation, covering development environment automation, issue triaging, and test coverage and quality assessment.

What the work involves

• Analyze and triage GitHub issues across trending open-source libraries.
• Set up and configure code repositories, including Dockerization and environment setup.
• Evaluate unit test coverage and quality.
• Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
• Collaborate with researchers to identify repositories and issues that are challenging for LLMs.

Who it's for

• Experienced software engineers (tech lead level) familiar with high-quality public GitHub repositories.
• Minimum 2+ years of overall experience.
• Strong experience with C++.
• Proficiency with Git, Docker, and basic software pipeline setup.
• Ability to understand and navigate complex codebases.
• Comfortable running, modifying, and testing real-world projects locally.
• Experience contributing to or evaluating open-source projects is a plus.

Nice to have

• Previous participation in LLM research or evaluation projects.
• Experience building or testing developer tools or automation agents.

Pay and schedule

• $25 per approved task.
• Minimum 40 hours per week, with 4 hours of overlap with PST.
• Contractor assignment (no medical or paid leave).
• Duration: 2-3 weeks; expected start date is next week.

Where applicants can be based

India, Pakistan, Nigeria, Egypt, Ghana, Bangladesh, Turkey, Mexico, or LATAM. The role is fully remote.

Selection process

Assessment-based review, approximately 75 minutes.