PowerShell Software Engineering Expert at Mercor

PowerShell Software Engineering Expert | $60–100/hr | Worldwide Remote

Mercor is seeking experienced PowerShell engineers and software professionals to evaluate and improve AI-generated code responses for leading AI development teams. This is a flexible contract role open to candidates globally, with both full-time and part-time arrangements available.

About the Role

Mercor partners with top AI organizations to enhance the quality, accuracy, and reliability of conversational AI systems. In this role, you will directly shape how AI models reason about code, generate solutions, and explain technical concepts — with a focus on PowerShell and broader software engineering domains.

What You'll Do

• Evaluate LLM-generated responses to coding and software engineering queries for accuracy, reasoning, clarity, and completeness
• Conduct fact-checking using trusted public sources and authoritative technical references
• Execute code and validate outputs using appropriate tools to ensure correctness
• Annotate model responses by identifying strengths, weaknesses, and factual or conceptual inaccuracies
• Assess code quality, readability, algorithmic soundness, and explanation quality
• Ensure model responses align with expected conversational behavior and system guidelines
• Apply consistent evaluation standards using clear taxonomies, benchmarks, and detailed evaluation guidelines

Who You Are

• Holds a BS, MS, or PhD in Computer Science or a closely related field
• 5+ years of real-world software engineering experience in technical roles
• Expert-level proficiency in PowerShell
• Able to independently solve HackerRank or LeetCode Medium and Hard-level problems
• Experience contributing to well-known open-source projects, including merged pull requests
• Significant experience using LLMs while coding, with a strong understanding of their strengths and failure modes
• Strong attention to detail and comfort evaluating complex technical reasoning

Nice-to-Have

• Prior experience with RLHF, model evaluation, or data annotation
• Track record in competitive programming
• Experience reviewing code in production environments
• Familiarity with multiple programming paradigms or ecosystems
• Ability to explain complex technical concepts to non-expert audiences

Why Join Mercor

This remote role lets experienced engineers apply deep technical expertise to high-impact AI development work. You'll play a direct role in improving AI systems relied upon by developers worldwide — all on a flexible schedule that fits your lifestyle.