Technical Reviewer - RL Benchmarking at Mercor

Technical Reviewer - RL Benchmarking | $80–100/hr | Remote in US

Join a leading AI research lab as a Technical Reviewer, where you'll play a vital role in evaluating and refining benchmarking pipelines for reinforcement learning (RL) environments and agentic AI systems. This is an exciting opportunity to ensure the accuracy, reproducibility, and fairness of experimental benchmarks that shape the future of intelligent agents.

• Key Responsibilities:
• Review RL environments and assess terminal conditions for correctness and consistency.
• Evaluate benchmarking pipelines for fairness, reproducibility, and research alignment.
• Provide structured technical feedback on code implementations and documentation.
• Collaborate with researchers to refine evaluation metrics and methodologies.
• Validate results across different runs, seeds, and hardware setups to ensure reproducibility.
• Document findings and recommend improvements for environment design and benchmarking standards.

Ideal Candidate:

• Background in reinforcement learning, computer science, or applied AI research.
• Experience with RL environments and benchmarking methodologies.
• Proficient in Python (PyTorch/TensorFlow a plus).
• Strong critical thinking and technical feedback skills.
• Detail-oriented, with a passion for experimental rigor and reproducibility.

Work Structure: Full-time hourly contractor, 40 hours/week, paid weekly via Stripe Connect. Flexible and remote work style.

Shape the standards of agentic AI research and collaborate with top researchers in a fast-moving field. Apply your expertise to advance the reliability and impact of RL benchmarking.