Design, develop, and optimize enterprise-scale data platforms, pipelines, data lakes, warehouses, and analytical datasets on Azure and Snowflake. Build ETL/ELT processes using ADF, Databricks, PySpark, SQL, Python, and Snowflake. Manage data models, governance, security, quality, CI/CD, cloud migrations, and performance optimization. Collaborate with stakeholders, troubleshoot production issues, document solutions, and support Agile delivery.
- Azure Data Engineer with 6+ years of experience designing, developing, and optimizing enterprise-scale data platforms using Microsoft Azure and Snowflake
- Proven expertise in building scalable data pipelines, data warehouses, and analytics solutions leveraging Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Snowflake, Python, SQL, and Spark
- Experienced in data modeling, ETL/ELT development, cloud migration, performance tuning, data governance, and implementing CI/CD practices
- Strong background in integrating data from multiple sources and enabling business intelligence and advanced analytics solutions
Key Skills: - Azure Data Factory (ADF)
- Azure Databricks
- Azure Synapse Analytics
- Snowflake Data Cloud
- Python, PySpark, SQL
- Azure Data Lake Storage (ADLS Gen2)
- Azure Functions
- Delta Lake
- Data Warehousing & Data Modeling
- ETL / ELT Development
- CI/CD (Azure DevOps, Git)
- Performance Tuning & Optimization
- Data Governance & Security
- Power BI Integration
- Agile & Scrum Methodologies
What you will do:
- Design, develop, and maintain scalable data engineering solutions on Azure and Snowflake platforms
- Build and optimize ETL/ELT pipelines using Azure Data Factory, Databricks, and Snowflake to ingest, transform, and load data from various structured and unstructured sources
- Develop and maintain enterprise data warehouses, data marts, and analytical datasets supporting reporting, BI, and advanced analytics initiatives
Implement data transformation and orchestration processes using PySpark, SQL, Python, and Snowflake stored procedures - Design and manage cloud-based data lakes using Azure Data Lake Storage (ADLS Gen2) and Delta Lake architecture
- Collaborate with business stakeholders, data analysts, and architects to gather requirements and translate them into scalable data solutions
- Monitor and optimize data pipeline performance, ensuring reliability, scalability, and cost efficiency
- Implement data quality frameworks, validation rules, and monitoring processes to ensure accurate and trusted data
- Manage Snowflake objects including databases, schemas, warehouses, tasks, streams, and data-sharing capabilities
- Develop and maintain dimensional and normalized data models following industry best practices
- Implement security controls, access management, encryption, and governance policies across Azure and Snowflake environments
- Support cloud migration and modernization initiatives from on-premises or legacy platforms to Azure and Snowflake
- Automate deployment processes using Azure DevOps, Git repositories, and CI/CD pipelines
- Troubleshoot production issues, perform root cause analysis, and provide timely resolution for data-related challenges
- Participate in Agile ceremonies, code reviews, and technical design discussions to ensure high-quality solution delivery
- Create technical documentation, architecture diagrams, operational runbooks, and knowledge-transfer materials
Preferred Certifications: - Microsoft Certified: Azure Data Engineer Associate (DP-203)
- SnowPro Core Certification
Similar Jobs
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Build and operate production data pipelines supporting analytics, AI, and agentic workflows. Responsibilities include implementing canonical data models, maintaining Databricks or Snowflake platforms, monitoring reliability, responding to incidents, validating healthcare data mappings, improving performance and cost, and documenting architecture. Requires strong SQL and Python skills, cloud data platform experience, ETL/ELT orchestration expertise, and healthcare or pharmaceutical data experience.
Top Skills:
Ai/Ml WorkflowsDatabricksEltETLHedisOmopPythonSnowflakeSQL
AdTech • Automotive • Big Data • Consumer Web
Administer and enhance Edmunds’ Databricks data platform and AWS infrastructure. Build and maintain ETL pipelines, infrastructure-as-code tooling using Terraform or CDK, and operational dashboards, alerts, and reports. Collaborate with business, engineering, analytics, security, and infrastructure teams to support data platform users and AI solutions. Evaluate new technologies, troubleshoot platform issues, and improve operational, cost, and security visibility.
Top Skills:
SparkAWSAws CdkDatabricksInfrastructure As Code (Iac)PythonScalaSQLTerraform
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Architects, deploys, and operates CrowdStrike’s data platform infrastructure and services. Responsibilities include administering Airflow and Superset, managing Terraform infrastructure, building CI/CD pipelines, implementing security and access controls, developing DBT data models, cataloging assets in OpenMetadata, writing Python automation, and mentoring engineers. The role requires cloud, container orchestration, SQL, data modeling, compliance, and AI technology expertise, along with eligibility for CJIS clearance.
Top Skills:
AIApache AirflowApache SupersetAWSAzureCi/CdDbtDockerGCPIamKubernetesMachine Learning PipelinesOciOpenmetadataPythonSQLTerraform
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
