This job post has expired on May 14, 2026. It is likely that the position has already been filled.
Agentic Coding Evaluation Specialist at Turing
posted 5 months agoAgentic Coding Evaluation Specialist | ~$75/hr | Remote (North America & Europe)
Join Turing as a contract Agentic Coding Evaluation Specialist and play a direct role in shaping the next generation of AI developer tools. You'll work with real-world codebases to design challenging prompts, run them across AI models, and evaluate outputs through structured comparisons — all contributing to measurable improvements in AI-assisted development workflows.
Key Responsibilities
- Analyze and navigate large open-source codebases such as React, Pydantic, and Pandas
- Design complex, single-turn coding prompts grounded in real repositories
- Execute prompts across multiple AI model instances and compare results
- Conduct side-by-side (SxS) evaluations of model outputs across key dimensions including:
- Instruction following
- Code quality
- Tool usage
- Testing & validation
- Communication clarity
- Provide clear, structured justifications for all evaluation decisions
Required Qualifications
- 2+ years of professional software development experience
- Bachelor's degree in Computer Science or a related field
- Strong proficiency in Python, JavaScript/TypeScript, or languages such as Java, Go, or C++
- Solid understanding of data structures & algorithms, debugging, testing practices, and software design principles
Engagement Details
- Rate: ~$75/hour
- Commitment: Minimum 20 or 40 hours per week (weekdays)
- Employment Type: Contractor (no medical/paid leave benefits)
- Duration: 1 week — expected start date April 17, 2026
- Eligible Locations: North America and Europe