Agentic Coding Evaluation Specialist at Turing

Agentic Coding Evaluation Specialist | ~$75/hr | Remote (North America & Europe)

Join Turing as an Agentic Coding Evaluation Specialist and play a direct role in shaping the future of AI-powered developer tools. You'll work with real-world codebases to design challenging prompts, run them across AI models, and evaluate outputs through structured comparisons — all contributing to smarter, more capable AI systems.

Engagement Details:

• Rate: ~$75/hour
• Commitment: Minimum 20 or 40 hours/week (weekdays)
• Employment Type: Contractor (no medical/paid leave benefits)
• Duration: 1 week (expected start: April 17, 2026)
• Location: North America or Europe (remote)

Key Responsibilities

• Analyze and navigate large open-source codebases (e.g., React, Pydantic, Pandas)
• Design complex, single-turn coding prompts grounded in real repositories
• Execute prompts across multiple AI model instances
• Conduct structured side-by-side (SxS) evaluations of model outputs
• Rate model performance across key dimensions including:
  • Instruction following
  • Code quality
  • Tool usage
  • Testing & validation
  • Communication clarity
• Provide clear, well-reasoned justifications for all evaluation decisions

Required Qualifications

• 2+ years of professional software development experience
• Bachelor's degree in Computer Science or a related field
• Strong proficiency in at least one of: Python, JavaScript/TypeScript, or languages such as Java, Go, or C++
• Solid understanding of data structures & algorithms, debugging and testing practices, and software design principles