Design and lead Databricks Lakehouse solutions, optimize data pipelines, implement Delta Lake practices, and integrate cloud services.
Company Description
Role: Databricks Architect
Duration: 12 months
Location: Troy, MI- Remote
Minimum exp.- 14-15 yrs.
Role Overview
We are looking for a Databricks Architect to design and lead modern Lakehouse data platforms using Databricks. The role focuses on building scalable, high-performance data pipelines and enabling analytics and AI use cases on cloud-native data platforms.
Key Responsibilities
- Architect and implement Databricks Lakehouse solutions for large-scale data platforms
- Design and optimize batch & streaming data pipelines using Apache Spark (PySpark/SQL)
- Implement Delta Lake best practices (ACID, schema enforcement, time travel, performance tuning)
- Build and manage Databricks jobs, workflows, notebooks, and clusters
- Enable data governance using Unity Catalog (access control, lineage)
- Integrate Databricks with cloud data services (ADLS / S3, ADF, Synapse, etc.)
- Support analytics, BI, and AI/ML workloads (MLflow exposure is a plus)
- Lead solution design discussions and mentor data engineering teams
Must-Have Skills
- 10+ years in data engineering / data architecture
- 5+ years of strong hands-on experience with Databricks
- Expert in Apache Spark, PySpark, SQL
- Strong experience with Delta Lake & Lakehouse architecture
- Cloud experience on Azure Databricks / AWS Databricks
- Proven experience in designing high-volume, scalable data pipelines
Good-to-Have
- Unity Catalog, MLflow, Databricks Workflows
- Streaming experience (Kafka / Event Hubs)
- CI/CD for Databricks (Azure DevOps / GitHub)
All your information will be kept confidential according to EEO guidelines.
Top Skills
Spark
Aws Databricks
Azure Databricks
Azure Devops
Ci/Cd
Databricks
Delta Lake
Event Hubs
Git
Kafka
Mlflow
Pyspark
SQL
Unity Catalog
Similar Jobs
.png)
Artificial Intelligence • Blockchain • Fintech • Financial Services • Cryptocurrency • NFT • Web3
Own the frontend experience of the Internal Developer Platform, build and maintain React components, and collaborate with cross-functional teams to enhance internal tools and improve user experience.
Top Skills:
DockerGoGraphQLNosql DatabasesReact
Artificial Intelligence • Blockchain • Fintech • Financial Services • Cryptocurrency • NFT • Web3
Develop and maintain mobile infrastructure, enhance developer tools, collaborate on features, implement app release automation, and mentor team members.
Top Skills:
AndroidExpoiOSJavaScriptReact NativeTypescript
Artificial Intelligence • Blockchain • Fintech • Financial Services • Cryptocurrency • NFT • Web3
The Product Manager II for DeFi Lend will lead the product strategy and execution for a new onchain lending service, coordinating closely with engineering and design teams to enhance user experience and manage risks effectively.
Top Skills:
AaveBlockchainCompoundCryptoCryptocurrencyDefiEthereumFinancial ProductsL2 NetworksMorphoSmart Contracts
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
