Posted over 30 days ago. Roles like this often stay open for a while, so it's still worth applying.
SWE-Bench AI Task Auditor at Invisible Tech
posted 1 month agoSWE-Bench AI Task Auditor | $70–$100/hr | Worldwide Remote | Freelance Contract
We are seeking experienced Software Engineering specialists to audit tasks used to train and evaluate AI systems. In this role, you will help ensure that AI training workflows are technically rigorous, practically sound, and highly accurate — playing a direct role in shaping the quality of next-generation AI models.
Project Deliverables & Scope
- Task Evaluation: Assess software engineering tasks for technical accuracy, realism, solvability, reproducibility, and the reliability of associated tests and evaluation criteria.
- Technical Auditing & Feedback: Identify and report on codebase integration issues, test failures, and logic errors — providing clear, actionable feedback to improve task quality.
Required Expertise
- Demonstrable professional experience and deep technical knowledge in software engineering, including complex codebase navigation, real-world application development, and familiarity with SWE-Bench standards. (Candidates will be evaluated for one specialty based on their background; cross-domain expertise is not required.)
- Strong analytical and problem-solving skills with the ability to rigorously test, troubleshoot, and evaluate complex technical scenarios.
Compensation & Details
- Pay Rate: $70–$100/hr, determined based on experience, expertise, and geographic location.
- Employment Type: Freelance Contract
- Workplace: Fully Remote
- Requirements: Contractors must supply their own secure computer and high-speed internet. Company-sponsored benefits such as health insurance and PTO do not apply.
If you have a strong software engineering background and a passion for improving AI systems, we encourage you to apply.
