Technical Reviewer - RL Benchmarking at Mercor

Technical Reviewer - RL Benchmarking | $25–30/hr | Remote

Join Mercor as a Technical Reviewer and play a pivotal role in advancing benchmarking standards for reinforcement learning (RL) environments and agentic AI systems. This position offers the opportunity to collaborate with leading AI researchers and engineers to ensure the accuracy, reproducibility, and fairness of RL benchmarking pipelines.

• Review and validate RL environments by assessing terminal conditions and evaluation protocols for correctness and consistency.
• Evaluate benchmarking pipelines to ensure fairness, reproducibility, and alignment with research objectives.
• Provide structured technical feedback on code implementations and documentation, working closely with research teams.
• Collaborate on refining evaluation metrics and methodologies to strengthen experimental rigor and system reliability.
• Ensure reproducibility by validating results across different runs, seeds, and hardware setups.
• Document findings and recommend improvements for environment design and benchmarking standards.

This role is ideal for candidates with a background in reinforcement learning, computer science, or applied AI research, who are experienced with RL environments and benchmarking methodologies. Strong critical thinking, attention to detail, and proficiency in Python (PyTorch/TensorFlow a plus) are essential. Enjoy flexible, full-time hours as a contractor, with weekly payments and the chance to influence the future of agentic AI research.