Terac Logo

Terac

AI Evaluators: Assessing A Shopping Assistant

Reposted 10 Days Ago
Remote
Hiring Remotely in United States
50-50 Hourly
Entry level
Remote
Hiring Remotely in United States
50-50 Hourly
Entry level
Evaluate conversations between users and an AI shopping assistant, identify logical errors, inaccuracies, and poor recommendations, and create structured rubrics and verifiers for judging response quality. This remote engagement requires at least 20 hours per week and focuses on improving model performance and e-commerce response quality.
The summary above was generated by AI
What We're Researching

We're hiring AI evaluators to assess the accuracy and helpfulness of a new digital shopping assistant. This project focuses on understanding how well the system handles real-world e-commerce queries and where it falls short in its logic. Your analysis will directly feed into improving the underlying model and its response quality.

How It Works

You will review real interaction traces between users and the shopping assistant within our custom platform. As you analyze these conversations, you will pinpoint specific failures, logical errors, or unhelpful product recommendations. From there, you will create structured rubrics and verifiers to consistently judge future response quality. This is an ongoing remote engagement requiring 20+ hours per week.

Who This Is For

This opportunity is ideal for quality assurance specialists, AI data evaluators, and e-commerce professionals with a strong eye for detail. We welcome applicants with prior experience in prompt engineering, complex data annotation, or software testing. You should be comfortable analyzing text interactions deeply and building structured evaluation frameworks from scratch.

What You'll Do
  • Review real user interaction traces with an AI shopping assistant

  • Identify logical failures, inaccuracies, or poor recommendations in the text

  • Create structured rubrics and verifiers to judge response quality

  • Commit to 20+ hours per week of evaluation work on our internal platform

Who Should Apply
  • Experience in data evaluation, quality assurance, or AI training

  • Strong analytical skills with the ability to spot subtle errors in text

  • Familiarity with e-commerce search and digital shopping experiences

  • Ability to commit to a sustained workload of 20+ hours per week

Compensation

$50 per hour

 
Ready to participate?

Start your paid interview now

 
About Terac

Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.

 

Learn more at terac.com or on YouTube at @jointerac.

Similar Jobs

An Hour Ago
Remote or Hybrid
45K-85K Annually
Junior
45K-85K Annually
Junior
Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Handles inbound calls and warm leads, consults customers on insurance needs, recommends appropriate coverages and products, and converts leads into policyholders. The role includes paid training and Property & Casualty licensing support, customer relationship management, sales conversion, and representing the Liberty Mutual brand. Employees work remotely on a fixed weekday and weekend schedule and must maintain a dedicated home workspace with reliable wired internet.
Top Skills: Cable InternetDsl InternetFiber InternetPcWired High-Speed Internet
An Hour Ago
Remote or Hybrid
125K-177K Annually
Senior level
125K-177K Annually
Senior level
Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Owns product vision, strategy, roadmap, backlog, prioritization, and delivery for a workers compensation transformation program. Partners with business and technology stakeholders, leads Agile ceremonies, defines actionable user stories, manages KPIs, resolves delivery barriers, evaluates costs and feasibility, and ensures compliance with industry constraints. Drives customer-focused design, usability testing, stakeholder alignment, and value delivery across squads and portfolios.
Top Skills: Agile
An Hour Ago
In-Office or Remote
29-52 Hourly
Junior
29-52 Hourly
Junior
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Coordinates nursing home-to-community transitions for Medicaid members. Supports applications, documentation, transportation, housing placements, household needs, and state reporting. Collaborates with members, care managers, nursing facilities, housing providers, and community organizations; monitors transition progress and ensures timely follow-up. Requires experience serving behavioral health or long-term-care populations, community provider knowledge, Microsoft Office proficiency, regional travel, a driver’s license, and reliable transportation.
Top Skills: ExcelMS OfficeMicrosoft OutlookMicrosoft Word

What you need to know about the Austin Tech Scene

Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.

Key Facts About Austin Tech

  • Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
  • Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
  • Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
  • Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
  • Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account