Create rigorous mathematics coding problems and verified Python solutions for AI research datasets. Develop unit tests and discriminative evaluations for AI-generated code, validate tasks using the Central Task Platform, refine work based on quality feedback, and optimize tasks for LLM evaluation criteria. The role requires mathematical accuracy, deterministic outputs, quality control, and participation in reviews and standups during overlap hours.
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
Role Overview
We are seeking a mathematics-focused professional to support an advanced AI research initiative developing high-quality STEM coding datasets. In this role, you will create rigorous mathematical problems, implement verified Python solutions, and develop evaluation tests designed to assess the capabilities of frontier AI models. The position combines mathematical problem-solving, scientific programming, and AI quality evaluation, with a strong emphasis on accuracy, determinism, and first-submission quality.
CONTRACT: Contractor assignment, 4 weeks
COMMITMENT: Hourly, up to 40h/week
LOCATIONS: Remote - Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Pakistan, Turkey, Vietnam
PROCESS: Interest Check Form, CV and Google Scholar review
Key Responsibilities
- Author mathematical coding tasks containing one main problem and at least 3 logically connected sub-problems that progressively build toward the main solution.
- Implement verified golden solutions in Python with complete unit test coverage.
- Design discriminative test cases that distinguish correct from incorrect AI-generated outputs.
- Perform quality control validation using the Central Task Platform (CTP), including Tier 1 structure checks and Tier 2 quality rubrics.
- Refine problem specifications, solutions, and test cases based on QC feedback.
- Optimize tasks to meet Pass@K evaluation criteria across multiple LLM judges, including GPT, Gemini, and Nemotron.
- Ensure mathematical correctness, well-posedness, determinism, and compliance with project quality standards.
- Maintain high first-submission quality and minimize rework, targeting consistent L1 approval.
- Participate in review sessions, feedback meetings, and project standups during required overlap hours.
Requirements
- Master's degree or PhD in Mathematics.
- Strong Python programming skills with experience in scientific computing.
- Experience using scientific Python libraries such as NumPy, SciPy, SymPy, or relevant domain-specific tools.
- Ability to formulate rigorous mathematical problems with clear constraints and expected outputs.
- Experience implementing scientific or mathematical solutions with unit tests and deterministic outputs.
- Prior experience in AI data annotation, scientific research, or scientific writing.
- Familiarity with LLM evaluation frameworks, coding benchmarks, or AI-generated code assessment.
- Published research or academic project experience in mathematics or another STEM discipline.
Similar Jobs
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Lead regional EHS for Network Operations across MOAI: set strategy, prevent serious injuries, govern psychosocial risks, manage workers' compensation, ensure environmental compliance, strengthen contractor governance, and develop a specialist team while influencing senior stakeholders and embedding EHS into commercial decisions.
eCommerce • Fashion • Retail • Sales • Wearables • Design
The Quality Auditor monitors manufacturing processes, conducts inspections, engages in root cause analysis, trains QC staff, and ensures quality compliance in footwear production.
Top Skills:
Quality Management
Automotive • Professional Services • Software • Consulting • Energy • Chemical • Renewable Energy
Support laboratory test setup, execution, and maintenance. Collect and analyze test data, prepare reports, assemble/disassemble test items, read technical drawings, operate lab equipment, and communicate findings to engineers while following safety and security protocols.
Top Skills:
ExcelMicrosoft PowerpointMicrosoft Word
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
