Author and verify rigorous multiple-choice legal assessment items for AI benchmarks across multiple legal specialties. Create clear, self-contained questions with one correct answer and nine plausible distractors, classify difficulty, provide structured solution explanations and references, and review/edit questions for legal accuracy and clarity. Work remotely, asynchronously, as an independent contractor, committing 10+ hours weekly.
This role is for one of our clients
Compensation: $83-$105 per hour
We are seeking experienced legal professionals to author and review high-quality academic assessment content for an AI research initiative. In this role, you will contribute your legal expertise to the development of rigorous evaluation benchmarks designed to assess how effectively AI systems understand and reason about complex legal concepts.
You may contribute through one of two task types:
Question Authoring: Create original, challenging multiple-choice questions within your area of legal expertise, assess their difficulty, and submit them for quality review.
Question Verification: Review existing questions for legal accuracy, clarity, completeness, and rigor. Make necessary edits, evaluate difficulty, and document the reasoning behind any changes.
RequirementsLaw Domains Covered
Opportunities span a broad range of legal specialties, including:
- Intellectual Property, Privacy & Technology
- Regulatory & Government Affairs
- Securities & Capital Markets
- Financial Regulation & Compliance
- Private Equity
- M&A & Transaction Structuring
- Antitrust, Merger Control & Competition
- Healthcare, Life Sciences & Pharmaceuticals
- Environmental, Energy & ESG/Climate Law
- Develop original legal questions that assess deep conceptual understanding, analytical ability, and legal reasoning rather than surface-level recall.
- Ensure every question is clear, unambiguous, self-contained, and sufficiently detailed to support a definitive answer.
- Classify questions by difficulty level: Medium (introductory undergraduate), Hard (advanced undergraduate), or Expert (postgraduate level and above).
- Provide one correct answer along with nine plausible but subtly incorrect alternatives designed to challenge advanced legal reasoning.
- Develop concise, logically structured solution explanations that demonstrate the appropriate legal reasoning process.
- Provide 1–5 credible academic or professional references for each question, drawing from reputable legal publications, academic sources, case law, statutes, and legal repositories.
- For verification assignments, identify issues involving legal accuracy, clarity, completeness, precision, or solvability and clearly explain any recommended revisions.
- Apply consistent quality standards across questions and contribute to the development of reliable, gold-standard legal evaluation benchmarks.
- JD, LLM, SJD, or doctoral-level qualification in Law or a closely related discipline.
- Master's-level candidates may be considered where they demonstrate exceptional expertise in a specialized legal domain.
- Strong command of legal reasoning, statutory and regulatory interpretation, case analysis, and legal research.
- Deep expertise in at least one of the covered practice areas.
- Experience with legal education, bar examination content development, academic assessment, judicial clerkships, legal research, or legal writing is highly valued.
- Excellent written English with the ability to communicate complex legal concepts clearly, precisely, and concisely.
- Strong attention to detail and the ability to identify subtle legal inaccuracies or ambiguities.
- Prior experience with AI evaluation, training data, benchmarking, or human-in-the-loop research is a plus.
- Commitment: 10+ hours per week
- Work Arrangement: Fully remote
- Work Style: Asynchronous and flexible
- Engagement: Project-based opportunities with potential for continued participation based on performance and project requirements
Interested candidates can submit a resume or relevant professional background highlighting their legal specialization and experience. Selected applicants may be asked to complete a short sample assessment or provide additional information about their subject-matter expertise.
Equal OpportunityWe consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Contract & Payment Terms- You will be engaged as an independent contractor.
- The role is fully remote and can be completed on a flexible schedule.
- Projects may be extended, shortened, or concluded based on project requirements and performance.
- Work will not require access to confidential or proprietary information belonging to any current or former employer, client, or institution.
- Payments are made weekly based on services rendered through available payment methods.
- H-1B and STEM OPT candidates are not eligible for this opportunity at this time.
Similar Jobs
Digital Media • Fintech • Information Technology • Machine Learning • Financial Services • Cybersecurity • Automation
Handle inbound and outbound mortgage collections calls, review account status, collect payments, evaluate customer finances, negotiate repayment solutions, and document account actions. Ensure compliance with customer information security, debt collection, and mortgage regulations while meeting quality, productivity, and operational targets. The role includes extensive training and operates on a hybrid schedule with three days onsite and two days remote in Irving, Texas.
Top Skills:
Ai SimulationsDigital Phone SystemsExcelMicrosoft WordMsp/Black Knight
Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Leads a specialized casualty claims team managing severe-exposure portfolios, complex litigation, coverage issues, reserves, settlements, vendors, and claim-handling protocols. Partners with legal, underwriting, actuarial, reinsurance, and other stakeholders to develop strategies, monitor litigation trends, address legal system abuse, manage financial outcomes, and improve operational performance. Coaches claims leaders, oversees staffing and portfolio capacity, and presents strategic recommendations to executives and business partners.
Top Skills:
Guidewire Claims System
Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Own model observability for Auto Physical Damage data science products, including traditional ML and GenAI/LLM systems. Build monitoring pipelines, dashboards, metrics, thresholds, alerts, and remediation documentation. Detect drift, performance degradation, anomalies, and data-quality issues; support MLOps, LLMOps, governance, compliance, and audit reporting. Analyze business outcomes, identify improvement opportunities, and communicate model health findings to technical and non-technical stakeholders.
Top Skills:
AWSAzureEmblemExcelGCPLlmopsMlopsPowerPointPythonSASSQLStreamlitTableauVBA
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
