This job post has expired on May 14, 2026. It is likely that the position has already been filled.
Agentic Coding Evaluation Specialist at Turing
posted 5 months agoAgentic Coding Evaluation Specialist | ~$75/hr | Remote (North America & Europe)
Join Turing as an Agentic Coding Evaluation Specialist and play a direct role in shaping the future of AI-powered developer tools. You'll work with real-world codebases to design challenging prompts, run them across AI models, and evaluate outputs through structured comparisons — all contributing to smarter, more capable AI systems.
Engagement Details:
- Rate: ~$75/hour
- Commitment: Minimum 20 or 40 hours/week (weekdays)
- Employment Type: Contractor (no medical/paid leave benefits)
- Duration: 1 week (expected start: April 17, 2026)
- Location: North America or Europe (remote)
Key Responsibilities
- Analyze and navigate large open-source codebases (e.g., React, Pydantic, Pandas)
- Design complex, single-turn coding prompts grounded in real repositories
- Execute prompts across multiple AI model instances
- Conduct structured side-by-side (SxS) evaluations of model outputs
- Rate model performance across key dimensions including:
- Instruction following
- Code quality
- Tool usage
- Testing & validation
- Communication clarity
- Provide clear, well-reasoned justifications for all evaluation decisions
Required Qualifications
- 2+ years of professional software development experience
- Bachelor's degree in Computer Science or a related field
- Strong proficiency in at least one of: Python, JavaScript/TypeScript, or languages such as Java, Go, or C++
- Solid understanding of data structures & algorithms, debugging and testing practices, and software design principles