Frontend Engineer – AI Code Evaluation at Mercor

Frontend Engineer – AI Code Evaluation | $90/hr | Worldwide Remote

Mercor is building a high-quality dataset of human preference judgments on AI-generated frontend code. This is evaluation work — not authoring. The core skill here is the ability to open someone else's page, understand how it was built, and identify exactly where it cuts corners.

What You'll Do

• Render a reference web page and two AI-generated candidate replications at a 1920×1080 viewport, then judge which is the closer reproduction — state by state.
• Evaluate visual fidelity in detail: box model, spacing, typography, color, borders, image handling, z-order, and overflow behavior.
• Read the source of both candidate attempts and grade construction quality — distinguishing a genuinely correct replication from one that merely looks correct at a single viewport.
• Test responsiveness across viewport widths and identify layout defects with precision.
• Write specific, evidence-cited justifications for every preference judgment. Vague rationale is not acceptable.
• Use the provided escape hatch appropriately when a task is genuinely uncompletable — neither over-flagging nor under-flagging.

Requirements

• 3–8 years of professional frontend or full-stack web development experience.
• Fluency across web eras: modern component frameworks and legacy table-based, float-based, and raw CSS layouts.
• Strong command of hand-written HTML and CSS: semantic markup, flexbox, grid, media queries, and legacy layout techniques.
• Browser DevTools as second nature — viewport control, element tree inspection, computed styles, hover state mutation.
• Command-line comfort: running a local static server, unzipping archives, resolving broken asset references.
• Enough JavaScript to read page scripts and understand their DOM interactions.
• Professional written English with the ability to produce clear, specific technical rationale.
• At least one completed Mercor engagement, delivered in full. Net-new experts are not being onboarded for this project.
• A desktop or laptop capable of displaying a true 1920×1080 viewport, with administrator rights to run a local server.

Nice to Have

• Prior RLHF, preference-labeling, model evaluation, or structured code-review experience.
• Pixel-perfect design-to-code background (agency work, design systems, template production).
• Accessibility expertise: ARIA, semantic landmarks, heading hierarchy.
• Familiarity with how LLMs fail at code generation.
• Web scraping, archiving, or DOM-parsing experience.
• Multiple completed Mercor projects and availability in contiguous multi-hour blocks.

Important Notes

Each unit takes approximately 2–3 hours and is timed. This is not microtask work — scattered 15-minute windows will not be sufficient to complete a unit. This role is specifically for practicing web developers. Backend-only, ML/data-science-only, mobile-native-only, DevOps-only engineers, and designers who do not code are not a fit for this position.