Build and optimize production batch and streaming data pipelines on Databricks using PySpark, SQL, Spark, and Delta Lake. Responsibilities include implementing Medallion architecture, deploying cloud data resources, automating validation, tuning performance, structuring enterprise datasets, and troubleshooting production issues. The role requires end-to-end ownership of enterprise data platforms, legacy warehouse migrations, data modeling, CI/CD, DevOps, and client-facing collaboration.
About V4C
Requirements
Benefits
V4C is a pure-play Databricks partner, and in under two years we have earned 850+ Databricks certifications across 325+ customers and been named Databricks' 2026 Partner of the Year (C&SI Emerging).
As a Databricks Engineer, you will build and optimize production pipelines on the Databricks Lakehouse, working directly with customers and often Databricks' own field teams. Every technical hire here gets a certification roadmap built for them and taught in-house by Databricks-approved trainers, plus a dedicated Employee Success Manager whose only responsibility is your experience at V4C.
PositionDatabricks Engineers build the pipelines enterprise customers run their business on. You will work under a senior engineer or architect, taking a defined piece of an engagement and owning it end to end: ingesting from systems that were not designed to be ingested from, transforming data through Medallion layers, and getting it to a state analysts and ML workloads can actually use. You will be client-facing — standups, working sessions, occasionally explaining to a stakeholder why something is harder than it looked. This is where most of our engineers start, and the certification roadmap is built around moving you up from it.
Responsibilities
- Build and maintain batch and streaming ETL pipelines using PySpark, SQL, and Delta Lake on Databricks
- Implement Medallion architecture layers
- Deploy and manage data resources on AWS, Azure, or GCP
- Write and automate data validation and quality checks
- Tune Spark jobs and SQL queries for performance and cost
- Structure and clean enterprise datasets for downstream analytics and AI workloads
- Troubleshoot pipeline failures and data quality issues, including in production
Requirements
- 8+ years in data engineering/architecture or analytics platform development
- Demonstrated ownership of end-to-end architecture for at least two enterprise-scale data platforms, from design through production
- Deep experience migrating legacy data warehouses or ETL estates to a modern cloud platform (Teradata, Informatica, Hadoop, Netezza, or equivalent)
- Strong proficiency in Python and SQL; working knowledge of Apache Spark and distributed processing fundamentals
- Experience with data modeling across dimensional, normalized, and Lakehouse patterns, and the judgment to know when each applies
- Experience with CI/CD and DevOps practices for data platform deployment
Preferences
- Healthcare or life sciences experience preferred but not required
- Databricks Certified Data Engineer Professional or a Platform Architect
- Hands-on Databricks experience at scale: Unity Catalog, Delta Lake, Lakeflow, Asset Bundles
- Experience with MLOps or productionizing machine learning workloads
- Prior professional services, consulting, or customer-facing architecture background
- Experience contributing to pre-sales scoping or technical proposals
Benefits
- Unlimited PTO
- Health Care Plan (Medical, Dental & Vision)
- Retirement Plan (401k, IRA)
- Life Insurance (Basic, Voluntary & AD&D)
- Family Leave (Maternity, Paternity)
- Short Term & Long Term Disability
- Training & Development
- Remote/WFH
Similar Jobs
Artificial Intelligence • Information Technology • Machine Learning • Generative AI • Big Data Analytics
Lead customer-facing data engineering engagements from architecture through production handover. Build and optimize batch and streaming Lakehouse pipelines on Databricks using PySpark, SQL, and Delta Lake. Establish Medallion architecture conventions, implement data quality and automated testing, optimize Spark performance and platform costs, and mentor engineers. Partner with architects and explain technical trade-offs to customers.
Top Skills:
AirflowSparkAWSAzureCi/CdDatabricksDatabricks Asset BundlesDbtDelta LakeGCPGitLakeflowPysparkPythonSQLUnity Catalog
Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Handle inbound calls and warm leads, consult customers on insurance needs, recommend appropriate coverages, and convert prospects into policyholders. Representatives receive paid training and licensing support, work remotely, and follow a schedule including one weekend day. The role requires strong communication, persuasion, organization, PC proficiency, and a Property & Casualty license after hire.
Top Skills:
PcWired High-Speed Internet
Automotive • Professional Services • Software • Consulting • Energy • Chemical • Renewable Energy
Leads GMP and quality system audits across manufacturing sites, farms, and processing facilities. Assesses compliance with pharmaceutical, supplement, medical device, cosmetics, food safety, and client standards; prepares detailed reports; mentors auditors; leads complex audits; and manages audit schedules and expenses. Requires frequent international and regional travel, strong analytical and communication skills, relevant credentials, and at least four years of related experience.
Top Skills:
21 Cfr 11121 Cfr 210/21121 Cfr 820CpgpCqaGfsiGmpIrcaIso 22716ExcelMicrosoft WordPcqiRabqsaRac
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
