Agentic Coding Evaluation Specialist at Turing

Agentic Coding Evaluation Specialist | ~$75/hr | Remote (North America & Europe)

Join Turing as a contract Agentic Coding Evaluation Specialist and play a direct role in shaping the next generation of AI developer tools. You'll work with real-world codebases to design challenging prompts, run them across AI models, and evaluate outputs through structured comparisons — all contributing to measurable improvements in AI-assisted development workflows.

Key Responsibilities

• Analyze and navigate large open-source codebases such as React, Pydantic, and Pandas
• Design complex, single-turn coding prompts grounded in real repositories
• Execute prompts across multiple AI model instances and compare results
• Conduct side-by-side (SxS) evaluations of model outputs across key dimensions including:
  • Instruction following
  • Code quality
  • Tool usage
  • Testing & validation
  • Communication clarity
• Provide clear, structured justifications for all evaluation decisions

Required Qualifications

• 2+ years of professional software development experience
• Bachelor's degree in Computer Science or a related field
• Strong proficiency in Python, JavaScript/TypeScript, or languages such as Java, Go, or C++
• Solid understanding of data structures & algorithms, debugging, testing practices, and software design principles

Engagement Details

• Rate: ~$75/hour
• Commitment: Minimum 20 or 40 hours per week (weekdays)
• Employment Type: Contractor (no medical/paid leave benefits)
• Duration: 1 week — expected start date April 17, 2026
• Eligible Locations: North America and Europe