Level Logo

Level

Applied AI Engineer

Posted Yesterday
Be an Early Applicant
In-Office
Austin, TX, USA
Senior level
In-Office
Austin, TX, USA
Senior level
Build and operate production agentic AI systems, including prompts, tools, retrieval, guardrails, evaluations, observability, and rollout. Automate expert workflows, ensure compliance and safety, and establish human-in-the-loop controls. Deliver documentation, runbooks, dashboards, and evaluation harnesses while enabling teams to adopt AI safely. The role requires substantial software engineering experience, production LLM deployments, retrieval expertise, structured outputs, local model experimentation, and strong communication.
The summary above was generated by AI

Applied AI Engineer
What You'll Do:

  • Ship production agentic systems- Design and build agents and agentic workflows that solve a defined problem end to end. You own prompts, tools, retrieval, guardrails, observability, cost and latency budgets, and rollout.

  • Provide governance and security guidance surrounding AI designs, strategy, and usage.

  • Automate SME workflows- Identify high-leverage operational toil (review pipelines, content QA, labeling and ops loops, support triage, internal copilots) and partner with the SMEs running those workflows to define success criteria, validate outputs, and replace meaningful chunks of that work with AI systems they trust. SMEs are co-owners of quality from the start of the project.

  • Make AI features safe- Treat what you deliver as a regulated product. Design for relevant compliance frameworks (e.g., COPPA, FERPA, NIST CSF 2.0) from day one, run safety and bias evals before launch and continuously after, and build the human-in-the-loop and content-filtering controls AI features need before they reach end users.

  • Hand off what you ship- Production features leave your hands with documentation, runbooks, an eval harness, and dashboards.

  • Make AI features easier for the teams to utilize- foster a culture of safe AI adoption with manageable ecosystems that integrate and enable cross-functional teams to create and enhance their own workflows and automations.

What You Need:

  • 7+ years professional experience as an Engineer with at least 1+ years of hands-on experience building agentic systems on at least one modern stack (LangGraph, the Anthropic SDK / Claude Agent SDK, OpenAI Agents SDK, Pydantic-AI, Mastra, LlamaIndex, CrewAI, or a homegrown stack). We care that you have built and operated agentic systems in production, not which framework.

  • Strong Python familiarity plus one typed language for production services (TypeScript, Go, or similar). Cloud experience (AWS or GCP) and containerized deployment.

  • Software engineering foundation (formal or autodidact), with several years of production environment experience and a track record of leading systems to launch.

  • Multiple shipped LLM-powered features in a production environment, with concrete stories about what broke, how you fixed it, and what you would do differently.

  • Practical knowledge of common agentic patterns: ReAct, tool use with structured schemas, prompt chaining, routing, orchestrator-workers, evaluator-optimizer / reflection, and human-in-the-loop. You can decide when a deterministic workflow is the right answer instead of an autonomous agent.

  • Hands-on experience in a production environment with retrieval: chunking, embeddings, hybrid search, re-ranking, metadata filtering, and the failure modes of each. Working knowledge of grounding techniques that anchor generated answers in retrieved evidence (citation and quote extraction, faithfulness and refusal evals, post-hoc consistency checks).

  • Strong prompt-engineering practice: zero-shot, few-shot, and many-shot patterns; example selection and ordering; in-context learning and chain-of-thought.

  • Comfort with structured output and validation in production (provider-native structured outputs, Instructor, Pydantic-AI, Outlines, or a comparable approach).

  • Disciplined evaluation practice. You do not rely on subjective review to decide whether a system is ready.

  • Experience in standing up local models and creating and testing closed local model solutions.

  • Strong written and verbal communication. You can explain an architectural trade-off to an executive and to a junior engineer in the same week.

  • You are comfortable using AI coding tools heavily in your implementation workflow while you own problem framing, design choices, and verification. We measure your output by working systems delivered, not lines of code written.

Nice to Have:

  • Advanced retrieval experience: GraphRAG, agentic retrieval, evaluation-driven retrieval tuning, and hybrid retrieval at scale.

  • Direct or transferable experience with safety, privacy, and policy constraints in user-facing AI. K-12 or other regulated-domain experience is a strong plus.

  • Experience with prompt-optimization frameworks (DSPy, TEXTGRAD, AdalFlow) where they paid off in production.

  • A public repo, package, gist, or technical write-up of meaningful AI work, or a representative project you can describe in detail under your confidentiality constraints.

  • Open-source contributions to AI tooling (frameworks, agents, evals, MCP servers).

Similar Jobs

3 Days Ago
Hybrid
Austin, TX, USA
124K-280K Annually
Senior level
124K-280K Annually
Senior level
Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Leads global teams developing production AI, GenAI, RAG, agentic workflow, and MLOps solutions for health systems and health plans. Translates clinical, financial, and operational challenges into HIPAA-compliant use cases involving clinical decision support, population health, revenue cycle, research, and optimization. Oversees healthcare data science engagements, model governance, executive stakeholder communications, client delivery, and team development.
Top Skills: Anthropic ClaudeAws BedrockAws SagemakerAzure Health Data ServicesAzure Machine LearningAzure Openai ServiceChromaCi/CdDatabricksGitGoogle Vertex AiKerasLangchainLlamaindexMlopsNoSQLPandasPineconePythonRagScikit-LearnSemantic KernelSnowflakeSQLTransformers
22 Days Ago
Hybrid
Austin, TX, USA
Senior level
Senior level
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Build production-grade platforms and governance tooling for AI-assisted software development. Responsibilities include developing reusable standards, registries, APIs, developer tools, AI marketplaces, security and intellectual-property integrations, and CI/CD governance. The role partners across engineering and security teams, takes ambiguous problems from concept through production adoption, and measures platform impact, reliability, and developer experience.
Top Skills: Agent FrameworksAIAPIsCi/CdCode OssLlmsPolicy-As-CodeVs Code
8 Days Ago
Hybrid
Austin, TX, USA
Senior level
Senior level
Hardware • Software
Design, build, deploy, and maintain AI-enabled applications and automation tools that improve engineering workflows. Develop LLM assistants, AI agents, document intelligence, developer productivity utilities, and enterprise integrations. Establish evaluation, monitoring, security, privacy, access-control, and responsible AI practices. Collaborate with engineering, security, IT, governance, and business teams to move AI prototypes into reliable production solutions.
Top Skills: Ai Application FrameworksC#Cloud ServicesData PipelinesEnterprise Ai PlatformsGitJavaJavaScriptLarge Language Model ApisPythonSvnTypescript

What you need to know about the Austin Tech Scene

Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.

Key Facts About Austin Tech

  • Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
  • Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
  • Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
  • Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
  • Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account