GenAI Engineer – Test Agent & CI at Turing

GenAI Engineer – Test Agent / CI Integration | Full-Time | Gurugram, India (Hybrid)

Join a forward-thinking team building intelligent, AI-powered testing systems for next-generation applications. This is a specialized role focused on automated test-agent frameworks, LLM evaluation, synthetic data generation, and CI/CD-integrated AI testing pipelines — not a general GenAI developer position.

Key Responsibilities

AI Test Agent Development

• Design and develop autonomous AI-driven test agents for validating GenAI and LLM-powered applications
• Build systems for synthetic data generation, test-case synthesis, scenario generation, and adversarial/edge-case testing
• Develop reusable evaluation harnesses for benchmarking model quality, accuracy, safety, and reliability

Context-Aware Test Generation

• Integrate test agents with knowledge/context graphs for retrieval-grounded testing
• Enable contextual test generation using RAG pipelines and graph-based retrieval systems
• Ensure generated tests align with enterprise knowledge sources and real-world workflows

CI/CD & Automation

• Integrate AI test agents into CI/CD pipelines as first-class pipeline jobs
• Automate regression testing, evaluation runs, and quality scoring during deployments
• Build scalable validation workflows for continuous model monitoring and release gating

Evaluation Frameworks & Quality Engineering

• Work with LLM evaluation frameworks including DeepEval, Ragas, and custom evaluators
• Develop automated scoring for hallucination detection, faithfulness, relevance, toxicity, and response quality
• Integrate with pytest and existing QA ecosystems

Backend & Infrastructure

• Build and maintain Python backend services powering evaluation workflows
• Optimize distributed evaluation execution for scalability and performance
• Collaborate with platform, MLOps, and DevOps teams for production deployment

Required Skills & Qualifications

• 4–8 years of experience in relevant engineering roles
• Strong proficiency in Python, pytest, REST APIs, and backend development
• Hands-on experience with CI/CD pipelines (GitHub Actions, Jenkins, GitLab CI, etc.)
• Experience with LLM evaluation frameworks (DeepEval, Ragas, LangSmith, or custom evaluators)
• Solid understanding of RAG systems, retrieval pipelines, and synthetic dataset generation
• Familiarity with vector databases, embeddings, and knowledge graphs
• Experience integrating AI workflows into CI/CD environments with automated quality gates

Preferred Qualifications

• Experience with knowledge graphs or graph databases
• Exposure to LangChain, LlamaIndex, or similar orchestration frameworks
• Familiarity with Kubernetes, Docker, and cloud platforms (AWS/GCP/Azure)
• Background in enterprise-scale AI platform engineering

This is a niche, high-impact role. Ideal candidates bring a unique blend of AI validation, QE automation, Python backend development, and LLM evaluation expertise.