Senior Software Engineer – Ruby at Turing

Senior Software Engineer – Ruby (LLM Evaluation) | Contractor | Remote – Select Countries

Join Turing, one of the world's fastest-growing AI companies, to help build LLM evaluation and training datasets for realistic software engineering tasks. This hands-on role blends practical Ruby engineering with cutting-edge AI research, working on verifiable software engineering tasks derived from real public repository histories.

About the Project

We are constructing LLM evaluation and training datasets using a synthetic, human-in-the-loop approach based on public GitHub repository histories. The goal is to expand dataset coverage across programming languages, difficulty levels, and task types to better train LLMs on real-world software engineering problems.

What You'll Do

• Analyze and triage GitHub issues across trending open-source libraries.
• Set up and configure code repositories, including Dockerization and environment automation.
• Evaluate unit test coverage and overall test quality.
• Run and modify codebases locally to assess LLM performance in bug-fixing scenarios.
• Collaborate with researchers to identify repositories and issues that challenge LLMs.
• Optionally lead a small team of junior engineers on collaborative tasks.

Required Skills

• 3+ years of overall software engineering experience.
• Strong proficiency in Ruby.
• Solid experience with Git, Docker, and software pipeline setup.
• Ability to navigate and understand complex, real-world codebases.
• Comfortable running, modifying, and testing projects in local environments.

Nice to Have

• Prior involvement in LLM research or evaluation projects.
• Experience building or testing developer tools or automation agents.
• Contributions to open-source projects.

Offer Details

• Commitment: 20, 30, or 40 hrs/week (minimum 4 hrs/day with 4-hour PST overlap).
• Employment Type: Contractor (no medical/paid leave benefits).
• Eligible Locations: India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico.

Evaluation Process (~75 minutes)

• Round 1: 60-minute technical interview.
• Round 2: 30-minute technical and cultural discussion.