Top Tech Jobs & Startup Jobs in Austin, TX

Reviews and compares AI chatbot responses using predefined criteria for accuracy, relevance, completeness, clarity, consistency, and quality. Documents evaluation guidelines, analyzes workflows and data, identifies recurring issues, prepares reports, tracks quality metrics, and collaborates with project teams. This fully remote, 16-week contract requires 8 hours per day, five days per week, with Pacific Time overlap and possible weekend work.

Evaluate educational AI conversations and generated responses for accuracy, relevance, completeness, clarity, and logic. Conduct research and fact-checking, synthesize information, annotate errors, provide evidence-based feedback, and create training data examples to improve AI models. The role requires reviewing genuine Gemini conversation history, submitting an exported JSON file, and working full-time remotely for a 16-week contract with PST overlap.

Design and validate complex engineering simulation problems for training and evaluating AI models and coding agents. Develop Python-based simulation workflows, automated tests, reference solutions, and grading systems across electrical, mechanical, and aerospace domains. Analyze AI execution logs and failure modes, validate physical plausibility and numerical convergence, refine benchmarks, and collaborate with engineering and AI research teams. This remote contractor role requires approximately 40 hours weekly, Pacific Time overlap, and weekend on-call availability.

Evaluates AI-generated data science and quantitative reasoning outputs; develops prompts, datasets, reference solutions, and objective evaluation benchmarks. Identifies statistical errors, flawed methodologies, and unsupported conclusions while providing structured feedback to improve model accuracy. Responsibilities include dataset validation, statistical and quantitative assessment, quality assurance, technical documentation, and clear communication of analytical findings.

Develop and evaluate backend security coding tasks for AI systems. Responsibilities include implementing backend features, fixing bugs, refactoring code, optimizing performance, identifying and remediating vulnerabilities, conducting security-focused code reviews, and creating deterministic automated tests and verifiers. The role requires strong backend engineering, application security, debugging, testing, algorithms, and software architecture skills. Work is remote, flexible, part-time, and contractor-based.

Develop and debug production-grade full-stack applications across front-end and back-end systems. Build features, APIs, databases, and data models; troubleshoot end-to-end issues; analyze unfamiliar codebases; translate implementations across programming languages; and review AI-generated and developer-written code for correctness, security, scalability, performance, and maintainability.

Develop and evaluate coding tasks for AI systems, including bug fixes, feature development, refactoring, optimization, reproducible environments, reference solutions, automated testing, and deterministic evaluation. The role also involves debugging complex codebases, reviewing contributions, documenting technical decisions, and refining evaluation criteria. Candidates should have strong software engineering fundamentals, backend or API development experience, open-source contributions, and proficiency in at least one listed programming language.

Develop coding tasks and reference solutions for evaluating AI systems. Build deterministic verifiers, test suites, scalable backend services, and APIs. Debug complex defects, assess code quality, refactor legacy implementations, optimize performance, and evaluate AI-generated technical reasoning. Document requirements and verification criteria while independently applying software engineering judgment across algorithms, system design, testing, and reliability.

Build and maintain software engineering infrastructure for AI development and evaluation workflows. Develop rubrics, review and label preference data, create scalable pipelines, document findings, navigate complex codebases, and improve evaluation quality and consistency. Collaborate with researchers and engineers in an ambiguous, fast-paced environment while applying strong judgment, debugging, communication, and system-design skills.

Perform contract drafting, review, redlining, and simulated negotiations while evaluating AI-generated legal responses. Identify contractual risks, assess legal accuracy using structured rubrics, and provide actionable feedback to improve AI systems for contract review. Collaborate with legal, product, engineering, and research teams on datasets, evaluation guidelines, and quality standards.