Goldman Sachs
Jobs
Senior Test Automation AI Engineer (Automation & Operations) - Vice President - Dallas
Goldman Sachs
Senior Test Automation AI Engineer (Automation & Operations) - Vice President - Dallas
Be an Early Applicant
Design and maintain AI-driven, self-healing test frameworks; implement LLM-powered test generation and agentic workflows; build telemetry and ML-based observability; orchestrate privacy-compliant synthetic data; integrate intelligent gates into CI/CD; and partner with engineers and data scientists to ensure testability of AI models and microservices.
Role Overview:
In the rapid development landscape of 2026, the role of a Senior AI/ML Engineer in test automation is to transform Quality Assurance (QA) from a reactive bottleneck into a proactive, intelligent layer. By leveraging Large Language Models (LLMs) and agentic workflows, you will build a "self-healing" test harness that provides the confidence needed for continuous, high-velocity deployments.
Responsibilities:
- Autonomous Test Harness Engineering: Design and maintain "self-healing" test frameworks that use AI to automatically update locators and scripts when UI or API schemas change, reducing maintenance toil by up to 70%.
- LLM-Powered Test Generation: Implement agentic workflows (using frameworks like LangGraph or CrewAI) to analyze Jira stories, PR diffs, and system architecture to generate comprehensive test suites, including edge cases and negative scenarios.
- Intelligent Observability & Monitoring: Build telemetry pipelines that use ML for anomaly detection and predictive risk analysis, identifying high-risk code areas before they reach production.
- Synthetic Data Orchestration: Leverage Generative AI to create high-fidelity, privacy-compliant synthetic datasets for complex integration and performance testing.
- "LLM-as-a-Judge" Implementation: Establish automated evaluation frameworks (e.g., Giskard, DeepEval) to measure the accuracy, safety, and hallucination rates of AI-driven features.
- CI/CD Integration: Architect intelligent gates within the CI/CD pipeline that use predictive test selection to run only the most relevant tests for a given code change, optimizing execution speed.
- Cross-Functional Collaboration: Partner with developers and data scientists to ensure "testability" is built into AI models and microservices from the design phase.
Similar Jobs
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Manage a team of TPRM analysts to run the vendor risk lifecycle, improve tooling and automation (ServiceNow TPRM, AI), perform assessments and audits, develop TPRM policies aligned to frameworks (NIST/ISO/SOC 2), partner with procurement/legal/IT, track KPIs, and support audit and reporting to leadership.
Top Skills:
Ai/Ml ToolsCloud EnvironmentsCrowdstrike ProductsFairIso 27001Nist 800-53Nist CsfSecure CodingServicenowServicenow TprmSigSoc 2
Machine Learning • Payments • Security • Software • Financial Services
Serve as a senior business systems analyst bridging business and IT: elicit and document requirements, design/configure system solutions, support integrations and API-based work, lead process improvements and testing (functional, integration, UAT), and provide senior consulting to improve business results.
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Manage supplier relationships, source selection, contract negotiation and execution, and supplier performance. Support risk management, supplier improvements, and integrate program/customer needs into contracting strategies. Ensure export control compliance and work onsite in San Antonio, coordinating schedules, quality, delivery, and financial supplier performance.
Top Skills:
Far
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center



