STEM Scientific Software Evaluation Design at Mercor

STEM Scientific Software & Evaluation Design | $45–$100/hr | Worldwide Remote

Join a cutting-edge project building large-scale evaluation benchmarks for advanced AI reasoning across scientific and engineering domains. As a Task Designer, you'll create graduate-level computational problems that challenge AI systems to use real scientific software tools — from querying simulations and interpreting outputs to designing experimental strategies and recovering hidden information from data.

This is not a typical annotation or labeling role. You'll be crafting original, research-grade problems, calibrating them against frontier AI models, and iterating until the difficulty hits the right target.

What You'll Do

• Design sophisticated computational problems requiring domain-specific scientific software libraries
• Create tasks that test precise multi-step scientific workflows as well as strategic experimental reasoning
• Participate in a calibration loop — testing problems against state-of-the-art AI models and refining designs accordingly
• Write problem setups, oracle functions, and solution validators in Python

Domains & Tools

• Bioinformatics & Single-Cell Genomics: scanpy, scvelo, squidpy, gudhi — RNA-seq, trajectory inference, spatial transcriptomics
• Computational Chemistry & Electronic Structure: PySCF — Hartree-Fock, DFT, TDDFT, CASSCF, post-HF methods
• Particle & Nuclear Physics: scikit-hep — HEP data analysis, cross-sections, perturbative QCD, Monte Carlo
• Electrical Engineering & RF/Circuit Design: scikit-rf, ngspice — S-parameters, transmission-line modeling, circuit simulation
• Astrophysics & Cosmology: astropy — cosmological calculations, angular power spectra, observational pipelines
• Structural & Mechanical Engineering: scikit-fem — finite element analysis, beam theory, elasticity problems
• Seismology & Geophysics: ObsPy, SPECFEM — waveform analysis, tomography, moment tensor inversion
• Pharmacokinetics & Systems Biology: libRoadRunner, Tellurium, SBML — PK/PD modeling, enzyme kinetics

Requirements

• Graduate-level training in a relevant STEM domain (MS, PhD, or equivalent research experience)
• Demonstrated hands-on proficiency with at least one listed scientific software library
• Strong Python programming skills
• Ability to work independently and iterate based on calibration feedback
• Comfortable in a Linux/terminal environment with remote compute sandboxes
• Available for at least 15–20 hours per week

Nice to Have

• Experience across multiple listed domains or tools
• Background in benchmark, evaluation, or exam/problem-set design
• Familiarity with computational reproducibility and containerized environments

Strong candidates think like puzzle designers — building problems where difficulty stems from reasoning strategy, not brute computation, and where surface-level pattern matching won't suffice.