AI Evaluation Engineer at Turing

AI Evaluation Engineer (Engineering Simulation & Design) | Contractor | 40 hrs/week | Worldwide Remote

Turing is seeking experienced AI Evaluation Engineers specializing in engineering simulation and design to author and validate complex, model-breaking problems that train and evaluate state-of-the-art AI agents. This is a high-impact contractor role working at the frontier of AI research across disciplines including Electrical, Mechanical, Control Systems, Aerospace, Systems, and Robotics Engineering.

About Turing

Based in San Francisco, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing accelerates frontier research through high-quality data, advanced training pipelines, and top AI researchers specializing in coding, reasoning, STEM, multilinguality, multimodality, and agents.

Key Responsibilities

• Model-Breaking Problem Design: Author original, self-contained engineering design tasks with competing constraints, explicit optimization targets, validated reference solutions, and objective autograders.
• Environment & Simulation Integration: Build, run, and validate problem environments using open-source simulation tools and custom Python test benches.
• Trajectory Analysis & Failure Mode Taxonomy: Evaluate coding agent outputs and execution logs across repeated trials to identify systemic failure modes such as misinterpreted simulator feedback, premature design convergence, or physically impossible geometries.
• Difficulty Calibration & Benchmark Refinement: Iteratively refine problem difficulty based on empirical model performance data without introducing ambiguity.
• Cross-Functional Collaboration: Partner with AI researchers, pod leads, and domain experts to integrate high-rigor benchmarks into the model evaluation pipeline.

Requirements

• Education: Master's degree or PhD in Electrical, Mechanical, Aerospace, Control Systems, Systems Engineering, Robotics, or an applied science discipline.
• Experience: 3+ years of hands-on engineering design experience.
• Simulation Tooling: Proficiency with at least one domain-relevant open-source simulation package (e.g., ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, OpenModelica, Cantera, Gmsh) and strong Python scripting skills.
• AI Evaluation: Hands-on experience with modern LLMs/coding agents and evaluation concepts (pass@k, failure-mode analysis, nondeterministic behavior), with the ability to audit trajectory logs and isolate core reasoning and tool-use failures.
• Domain Rigor: Uncompromising attention to physical plausibility, unit consistency, boundary conditions, convergence criteria, and technical documentation.
• Availability: Weekend on-call availability required (part-time engagement acceptable).
• Infrastructure: Personal desktop or laptop with a stable, high-speed internet connection.

Target Engineering Domains

• Electrical Engineering
• Mechanical Engineering
• Aerospace Engineering

Engagement Details

• Commitment: 40 hours per week with 4 hours of overlap with PST
• Engagement Type: Contractor
• Duration: Up to 24 weeks

Evaluation Process

• Shortlisted candidates will receive a Job Interest Form.
• Finalized candidates will undergo a delivery review before proceeding further.