LLM DevOps Engineer at Turing

LLM DevOps Engineer | Contractor | Fully Remote | Turing

Join Turing — the world's leading AI research accelerator — and contribute to improving frontier Large Language Models (LLMs) through high-quality data generation, model evaluation, and infrastructure expertise. This is a short-term contractor role (1 month) working directly with a foundational LLM company, with potential for extension based on performance.

About Turing

Based in San Francisco, Turing partners with leading AI labs and global enterprises to accelerate frontier AI research and deploy advanced AI systems at scale. Turing's work spans coding, reasoning, STEM, multilinguality, multimodality, and agents.

Role Overview

This role focuses on generating high-quality proprietary data to help foundational LLM companies fine-tune and benchmark their models. You will design DevOps-focused prompts, implement verification scripts, conduct model evaluations, and contribute to Supervised Fine-Tuning (SFT) and Reinforcement Learning with Human Feedback (RLHF) pipelines. Note: This role does not require building or fine-tuning LLMs directly.

Day-to-Day Responsibilities

• Design and develop challenging, technically rigorous prompts covering DevOps and infrastructure technologies.
• Implement executable verification code to validate model responses to prompts.
• Conduct evaluations (Evals) to benchmark model performance and analyze results for continuous improvement.
• Evaluate and rank AI model responses across diverse domains, ensuring alignment with predefined quality criteria.
• Develop detailed explanations and rationales for evaluations, demonstrating strong technical reasoning.
• Lead Supervised Fine-Tuning (SFT) efforts, including creating and maintaining high-quality, task-specific datasets.
• Collaborate with researchers and annotators on RLHF workflows and reward model refinement.
• Design innovative evaluation strategies to improve model alignment with user needs and ethical guidelines.
• Conduct thorough peer reviews of code and documentation, providing constructive feedback.
• Collaborate cross-functionally to improve model performance and contribute to product enhancements.

Requirements

• Technical Expertise:
  • Proven experience with configuration management and infrastructure automation tools such as Ansible, Terraform, CloudFormation, or similar platforms.
  • Strong exposure to AWS cloud platforms with experience designing and managing multi-cloud environments.
  • Hands-on experience with Docker and Kubernetes for containerization and orchestration.
  • Proficiency in scripting languages such as Bash and Python for automation and tool integration.
  • Familiarity with CI/CD tools (Jenkins, GitLab CI, CircleCI, etc.) and version control systems (Git).
• Operational Excellence:
  • Experience setting up monitoring, logging, and alerting mechanisms for system health and incident response.
  • Knowledge of networking, security best practices, and high-availability design in cloud infrastructures.
• Professional Skills:
  • 5+ years of overall work experience in DevOps or related roles.
  • Strong ability to collaborate with cross-functional teams and communicate complex technical concepts clearly.
  • Proactive problem-solving skills with a focus on identifying and resolving system bottlenecks and vulnerabilities.
  • Fluent in conversational and written English.

Engagement Details

• Commitment: Minimum 4 hours/day, 20 hours/week, with a 4-hour overlap with PST.
• Employment Type: Contractor (does not include medical or paid leave benefits).
• Duration: 1 month, with potential for extension based on performance and project needs.
• Environment: Fully remote.

Perks of Freelancing With Turing

• Work in a fully remote environment from anywhere in the world.
• Opportunity to contribute to cutting-edge AI projects with leading technology companies.
• Potential for contract extension based on performance and project requirements.