This project focuses on the technical review and validation of coding tasks used to train AI models. Coding challenges are generated automatically by AI, and your role is to evaluate quality of the challenge.
The core questions you will be answering on every task:
Are the requirements well defined?
Are the associated tests correctly validating the requirements?
Tests are treated as the mechanism of truth in this context. Mistakes here propagate at scale into AI systems — error criticality is high.
LanguagesTasks in this project involve code written across multiple languages. You will be expected to review and evaluate tasks in any of the following:
Python
JavaScript / TypeScript
Go
Rust
Java
Evaluate whether the coding task itself is clearly and correctly defined
Validate whether tests accurately reflect whether the problem has been solved
Determine whether a solution that passes the tests genuinely solves the underlying problem
Apply strict technical criteria and quality standards consistently across tasks
This role is designed for mid/senior-level software engineers with real professional experience.
Technical requirements
Proven professional experience in software development (production environments)
Experience reviewing and evaluating code written by other engineers
Solid understanding of automated testing — how tests validate (or fail to validate) behavior
Experience contributing to or working with open source projects
High attention to detail and strong technical judgment
Comfortable working fully in English (reading and writing)
Nice to have
Experience with test-driven development (TDD) or test design
Familiarity with large or complex codebases
This is expert, task-based technical work focused on analysis, validation, and judgment — not code production.
CompensationUSD 60-80 per hour.
~5-10 hours per week.
Ready to participate?
Start your paid interview now
About Terac
Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.
Learn more at terac.com or on YouTube at @jointerac.
Similar Jobs
What you need to know about the Austin Tech Scene
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center



