Leads large-scale AI and software-engineering data programs, owning delivery scope, timelines, quality, throughput, contributor performance, and cost. Designs workflows, quality-control systems, metrics, and capacity plans for coding datasets, agentic trajectories, benchmarks, and evaluations. Manages distributed contributor and reviewer networks, partners with talent teams, communicates with AI-lab customers, analyzes operational data, and converts successful processes into reusable playbooks and evaluation assets.
About Gramian
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
About the Role
We are looking for an experienced Technical Program Operations Lead to own the production systems behind large-scale AI and software-engineering data programs. You will turn complex research requirements into predictable delivery across quality, throughput, contributor performance, timelines, and cost.
Programs may include coding datasets, repository-level tasks, agentic trajectories, reinforcement-learning environments, benchmarks, code reviews, and rubric-based evaluations. This is an operations leadership role with a strong technical bar, requiring the ability to inspect code, understand tests, analyze quality signals, and improve workflows at scale.
Key Responsibilities
- Own end-to-end program delivery across scope, timelines, quality, throughput, contributor performance, and cost.
- Design and manage workflows for coding datasets, agentic trajectories, RL environments, benchmarks, and rubric-based evaluations.
- Identify operational bottlenecks and improve workflows through better instructions, sequencing, incentives, review systems, and capacity planning.
- Define contributor requirements and partner with talent teams to source, assess, onboard, train, and ramp distributed software engineers.
- Build team-lead and reviewer structures for programs involving 100–1,000+ contributors.
- Own quality-control systems and analyze datasets to identify trends, systematic errors, and root causes.
- Act as a primary customer contact for AI labs, communicating progress, risks, quality trends, and recovery plans.
- Translate research objectives into practical task specifications and challenge requirements when they may not produce the intended evaluation signal.
- Use Python, SQL, or similar tools to automate quality sampling, defect analysis, throughput reporting, and operational reviews.
- Convert successful workflows into reusable playbooks, quality controls, evaluation assets, and contributor-management systems.
- Share operational learnings and mentor other program leads.
Requirements
- Proven experience leading complex, multi-stakeholder programs in software engineering, technical program management, consulting, finance, startups, operations, or a similar environment.
- Strong analytical and problem-solving skills, including the ability to identify bottlenecks, define meaningful metrics, and improve production performance.
- Experience managing distributed teams, contributor networks, marketplaces, or large-scale technical operations.
- Strong customer-facing communication skills, including managing expectations, communicating risks, and building long-term client relationships.
- Ability to read and review code, understand test suites, and independently assess technical work.
- Working knowledge of at least one programming language such as Python, TypeScript, Java, or Go.
- Experience using data and operational metrics to monitor quality, throughput, performance, and delivery.
- Ability to operate effectively in environments where research requirements and priorities evolve quickly.
Similar Jobs
Agency • Gaming • Marketing Tech • Mobile • Analytics
Manages packaging supply chain operations across assigned markets, including category strategy execution, strategic sourcing, supplier relationship management, negotiations, contracts, supplier performance, supply assurance, risk mitigation, cost optimization, and supply planning. Partners with product development, quality, commercial, procurement, and suppliers to ensure continuity, quality, service, innovation, and financial performance. Requires strong commercial, analytical, stakeholder management, and procurement expertise.
Top Skills:
MS OfficeProcurement SystemsSourcing Systems
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Promotes hematology pharmaceutical products to healthcare professionals, manages a geographic territory, builds relationships with key opinion leaders, and achieves sales targets. Responsibilities include presenting clinical data, executing promotional strategies, monitoring competitors and market trends, maintaining compliant customer records, preparing reports, and using omnichannel and digital sales tools. The role requires hematology experience, medical representative experience, scientific knowledge, strong selling and presentation skills, and a pharmacy or related scientific degree.
Top Skills:
ExcelMicrosoft OutlookMicrosoft PowerpointMicrosoft WordPfizer SoftwareSales Analytics Platforms
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Designs and supports IP core and transport network solutions for customer projects. Responsibilities include requirements analysis, high-level architecture, integration planning, troubleshooting, testing, acceptance, migration, technical documentation, stakeholder coordination, customer workshops, and operational handover. The role also supports proposals and solution estimates while contributing to network automation, knowledge sharing, and Ericsson solution reuse.
Top Skills:
BgpCcdCcnaCcnpCiscoCloud InfrastructureCnisEricsson Ip NetworkingEricsson PlatformsEvolved Ip NetworkHuaweiIp NetworkingIp/MplsIs-IsJncipJuniperL2/L3 VpnsNetwork AutomationNetwork ResiliencyNetwork SecurityNetwork SynchronizationNfviNokiaObservabilityOspfQosRoutingScriptingSwitchingTelecommunicationsTelemetryTraffic Engineering
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
