Construction Estimating AI Evaluator at Mercor

Construction Estimating AI Evaluator | $50/hr | Remote (US & Canada)

Join a cutting-edge benchmark dataset project focused on evaluating AI models for visual document understanding and instruction-following within the Construction Estimating domain. This is an exciting opportunity for industry experts to directly shape the future of AI capabilities in a specialized field.

About the Role

As a subject matter expert, you will author complex, grounded tasks designed to test AI model performance. Each task must include a clear ground-truth output and an objective evaluation rubric, ensuring rigorous and meaningful benchmarks.

Key Responsibilities

• Design and author challenging visual document understanding tasks rooted in construction estimating workflows
• Develop clear ground-truth outputs and objective scoring rubrics for each task
• Evaluate AI model responses for accuracy, instruction-following, and domain relevance
• Provide structured feedback to improve model performance in the construction domain

Requirements

• Demonstrated expertise in construction estimating (e.g., quantity takeoffs, bid preparation, cost analysis)
• Strong attention to detail and ability to define objective evaluation criteria
• Comfortable working with visual documents such as blueprints, specs, and cost sheets
• Based in the United States or Canada

Engagement Details

• Rate: $50/hr
• Hours: ~15–20 hrs/week
• Location: Fully remote
• Type: Part-time contractor engagement

This is a private shortlist opportunity and is not publicly listed on the general job board.