Engineering Manager – LLM Evaluation at Turing

Engineering Manager – LLM Evaluation | Contract | Fully Remote | Min. 40 hrs/week

Turing is seeking an experienced Engineering Manager to lead teams building high-quality conversational datasets for training and evaluating advanced Large Language Models (LLMs). This is a fully remote contract role with a minimum 40-hour weekly commitment and at least 4 hours of daily overlap with PST.

About Turing

Turing is a leading AI company accelerating the advancement and deployment of frontier AI systems. We partner with the world's top AI labs and enterprises to push the boundaries of reasoning, coding, agentic behavior, multimodality, and other advanced AI domains.

What You'll Do

• Lead and manage teams responsible for creating and evaluating multi-turn LLM conversations.
• Define workflows, priorities, quality standards, and delivery goals for conversation data projects.
• Guide the design of realistic assistant interactions involving APIs, JSON, function calls, and external tools (calendars, email, maps, cloud storage, etc.).
• Review complex scenarios covering decision-making, clarification, error handling, constraints, and task completion.
• Ensure conversations accurately reflect realistic user expectations and assistant capabilities.
• Establish quality-control processes and monitor team performance, consistency, and deliverable quality.
• Coach team members on technical reasoning, conversation design, tool usage, and project guidelines.
• Collaborate with engineering, product, operations, and quality teams to resolve issues and improve workflows.
• Analyze reviewer feedback and implement process improvements to increase realism, accuracy, and training value.
• Manage project risks, timelines, escalations, and stakeholder communication.

What We're Looking For

• 5+ years of professional experience in software engineering, technical operations, data, AI, or a related technical field.
• Prior experience managing, mentoring, or leading technical teams.
• Strong understanding of APIs, JSON, data formats, logical reasoning, and structured workflows.
• Ability to guide teams in breaking down complex tasks into realistic conversational and tool-use flows.
• Deep understanding of assistant behavior, including tool selection, clarification, limitations, and error handling.
• Excellent written and verbal English communication skills.
• Strong attention to detail with the ability to establish and enforce quality standards.
• Proven ability to manage multiple priorities, provide actionable feedback, and deliver projects on time.
• Effective collaborator across cross-functional teams and with external stakeholders.

Offer Details

• Commitment: Minimum 40 hours/week with at least 4 hours daily overlap with PST.
• Duration: 3-month contract (adjustable based on engagement).
• Environment: Fully remote.

Perks of Freelancing With Turing

• Work in a fully remote, flexible environment.
• Opportunity to contribute to cutting-edge AI projects with leading LLM companies worldwide.