AI Evaluations Program Manager

The role:

We are seeking an AI Evaluations Program Manager to own the end-to-end process for evaluating AI chat outputs in a financial services context. This role is critical to ensuring our AI delivers guidance and education that is accurate, compliant, consistent, and genuinely helpful to members who want to Get Their Money Right. 

You’ll be responsible for managing the daily evaluation workflows for AI-generated chat responses, with particular attention to financial accuracy, regulatory risk, and user helpfulness. You’ll also own the continuous improvement loop, translating evaluation insights into measurable improvements through partnership with our Legal, Compliance, Engineering, Product, and Design (EPD) teams. 

As the program manager, you’ll sit at the intersection of product, AI, operations, risk, and financial expertise, helping us achieve our mission to empower members to reach financial independence.

What you’ll do:

• Own the AI Evaluation Program on SoFi’s AI powered financial coach, designing processes related to daily workflows, schedules for grading and reporting. 

• Work alongside our technology team to integrate feedback and test improvements of AI guardrails and overall knowledge base. 

• Establish and lead the continuous improvement loop to ensure all stakeholders are informed of performance, improvements, trends, issues and other metrics.  

• Partner with Product, Legal & Compliance to maintain the AI knowledge base as well as any data retention and documentation processes. 

What you’ll need:

• 8+ years of professional experience working in program management, operations, quality, risk, or evaluation roles within a financial services organization. 

• Familiarity with AI/LLM technologies and their application in providing personalized guidance.

• Experience building and managing structured processes in complex or regulated environments. 

• Excellent analytical skills and comfort working with both qualitative and quantitative data. 

• Strong Stakeholder Management skills, with the ability to drive alignment across technical, financial, and regulatory partners. 

• Familiarly with financial products such as investments, retirement accounts, loans and banking. 

Nice to have:

• Credentials: CFP® or other professional financial designation is a ✓++

• Licenses: FINRA Series 65 or interest in obtaining 

• Prior experience in financial technology (fintech), Financial Advisory or Wealth Management. 

• Experience directly working with technical teams