Senior C++ Engineer – LLM Evaluation at Turing

Senior C++ Software Engineer – LLM Evaluation & Repository Validation | Contractor | Remote (India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico)

Join Turing — one of the world's fastest-growing AI companies — and help shape the future of AI-assisted software development. This hands-on contractor role focuses on building high-quality LLM evaluation and training datasets using real-world open-source repositories, with a human-in-the-loop approach.

About the Project

We are constructing verifiable software engineering tasks derived from public GitHub repository histories. The goal is to expand dataset coverage across programming languages, difficulty levels, and task types — ultimately training LLMs to solve realistic engineering problems.

What You'll Do

• Analyze and triage GitHub issues across trending open-source C++ libraries.
• Set up and configure code repositories, including Dockerization and environment automation.
• Evaluate unit test coverage and overall test quality.
• Run and modify codebases locally to assess LLM performance in bug-fixing scenarios.
• Collaborate with AI researchers to identify repositories and issues that challenge LLMs.
• Optionally lead a small team of junior engineers on collaborative tasks.

Required Skills

• 3+ years of professional software engineering experience.
• Strong proficiency in C++.
• Solid experience with Git, Docker, and software pipeline setup.
• Ability to navigate and understand complex, real-world codebases.
• Comfortable running, modifying, and testing projects in local environments.

Nice to Have

• Prior involvement in LLM research or evaluation projects.
• Experience building or testing developer tools or automation agents.
• History of contributing to or evaluating open-source projects.

Engagement Details

• Type: Contractor (no medical/paid leave benefits)
• Commitment: Minimum 20 hrs/week (options: 20, 30, or 40 hrs/week); at least 4 hrs/day with 4-hour PST overlap required.
• Location: India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico

Evaluation Process (~75 minutes)

• Round 1: 60-minute technical interview
• Round 2: 30-minute technical & cultural discussion