Lead GPU-edge inference systems, ensuring compliance and performance through design and monitoring. Manage global infrastructure for AI deployment.
Full-time | Remote | Infrastructure | Reports to CTO
About Elloe
Elloe is the trust layer for AI.
We sit between the world’s most powerful language models and the institutions that can't afford to get it wrong — hospitals, banks, regulators. We trace and block failures in real time. That’s not marketing — we’re deployed at the European Commission, with NIH clinical trials, and inside a Top-5 EU bank catching GDPR violations live.
This is the enforcement layer GenAI has been missing. We're not visualizing problems — we're fixing them.
About the Role
You’ll lead our GPU-edge inference systems. From chaos-resilient deployment to SHAP-driven compliance metrics, you’ll own global infra that makes AI safe and performant.
What You’ll Own
1. Global Edge Routing
- Design zone-routing that ensures <50ms SLA in 10+ regions
- Build fallback orchestration to handle compliance-aware rollbacks
2. GPU Infra Ops
- Maximize utilization across 100K+ GPUs via mesh & load prediction
- Integrate compliance overlays with VaultChain and SHAP triggers
3. Reliability Telemetry
- Ship `/vault/audit`, `/inference/predict`, `/compliance/log` endpoints
- Trace every edge request across governance and model layers
Who You Are
- Senior systems engineer with GPU fleet experience (KubeRay, Istio, Envoy)
- Operated real-time AI infra with 10M+ QPS loads
- Comfortable with compliance observability and infra governance
Why This Matters
Our competitive edge isn’t just AI — it’s defensible enforcement. This role turns that into product.
You’ll Leave This Role With
- Referenceable contributions to enforcement infra that’s live in EU and US institutions
- First-hand product work across legal, engineering, and GTM teams
- Influence over how regulatory primitives become systems people trust
Logistics & Application
- Start Date: Flexible (Q3–Q4 ideal)
- Location: Remote-first; timezone overlap with NY or EU preferred
- Compensation: Top of market salary + equity
- To Apply: Send your resume and a sentence on the hardest infra problem you'd want to own at scale.
Elloe AI Austin, Texas, USA Office
701 Brazos St, Austin, Texas, United States, 78701 3258
Similar Jobs
Other
Provides field-office operational support across procurement, warehousing, inventory, fleet, assets, facilities, travel, and administration. Processes purchases, manages suppliers and records, oversees stock and warehouse compliance, coordinates vehicles and fuel reporting, maintains asset accountability, and supports visitors, events, accommodations, and office services. The role ensures compliance with organizational policies, donor requirements, health and safety standards, and audit expectations while contributing to efficient programme delivery.
Top Skills:
Erp SystemsGlidegoExcelMS OfficeMicrosoft OutlookMicrosoft WordProsave
Fintech • Payments • Financial Services
Develop and execute group sourcing strategies for smartphones, feature phones, and accessories. Lead tenders and negotiate pricing, volumes, contracts, logistics, warranty, payment terms, and foreign-exchange exposure with OEMs, ODMs, and distributors. Manage supplier performance, quality, supply risks, audits, counterfeit risks, and type approvals. Maintain audit-ready procurement records, report savings and device costs, and lead a team of three.
Top Skills:
Erp SystemsMicrosoft DynamicsExcelProcurement Systems
Healthtech • Biotech • Pharmaceutical • Manufacturing
Leads biostatistical work for clinical trials, including study design, protocols, analysis plans, statistical methodology, output interpretation, and regulatory or publication deliverables. Collaborates with programming, data management, clinical teams, KOLs, and senior statisticians, with a focus on exploratory analyses in cardiovascular and chronic kidney disease therapeutic areas. Provides peer review, technical guidance, and mentoring on complex biostatistical projects.
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center

