Datavail Logo

Datavail

Senior Data Engineer

Posted 19 Days Ago
Remote
Hiring Remotely in USA
Senior level
Remote
Hiring Remotely in USA
Senior level
Designs and leads production-grade batch, incremental, streaming, and real-time data pipelines across Microsoft Fabric and Azure. Builds Delta Lake medallion architectures, dimensional and Data Vault models, CDC and incremental-loading solutions, CI/CD automation, governance, security, observability, and data-quality controls. Supports migrations to Fabric, prepares governed data for AI and RAG workloads, and leads client-facing engineering workstreams, estimates, risk management, mentoring, and operational handover.
The summary above was generated by AI
  • 8+ years in data engineering/data warehousing; 3+ years delivering Azure solutions and production
  • Microsoft Fabric experience
  • Advanced T-SQL, Spark SQL, Python/PySpark; working knowledge of KQL and DAX
  • Deep experience with Delta Lake, Spark tuning, medallion/lakehouse patterns, CDC and incremental loading
  • Strong dimensional modeling and working knowledge of Data Vault 2.0
  • Practical CI/CD, Git, automated testing, infrastructure as code, DataOps, governance and security experience
  • Professional-services/client-facing delivery experience
  • Authorization to work in the US and ability to travel up to 15%
  • Preferred: DP-700, DP-600, DP-203; streaming/event-driven, RAG/GenAI, Fabric IQ/data agents, and industry experience
  • Design, build and maintain production-grade batch, incremental, real-time and near-real-time data pipelines across Microsoft Fabric and Azure
  • Implement CDC, watermarking, merge/upsert, late-arriving-data and idempotent reprocessing patterns
  • Develop PySpark, Spark SQL and T-SQL transformations across bronze, silver and gold Delta Lake layers
  • Build Fabric solutions using OneLake, Lakehouse, Warehouse, Data Factory, Dataflows Gen2, Spark notebooks, Real-Time Intelligence and Power BI
  • Implement dimensional and Data Vault models, data-quality testing, reconciliation, observability, logging, alerting and SLA monitoring
  • Establish Git-based CI/CD and automated deployment using Azure DevOps or GitHub, Fabric deployment pipelines, REST APIs, Bicep and Terraform
  • Apply Purview catalog/lineage, least-privilege access, sensitivity controls, encryption and secure connectivity
  • Support migrations from SQL Server, SSIS, SSAS, legacy warehouses and Synapse to Fabric/Azure, including conversion, reconciliation and cutover
  • Prepare governed data for AI/ML and RAG workloads, including ingestion, chunking, embeddings, indexing and telemetry
  • Lead engineering workstreams, estimates, risk management, client communication, mentoring, reusable accelerators and operational handover
About UsDatavail is a leading provider of data management, application development, analytics, and cloud services, with more than 1,000 professionals helping clients build and manage applications and data via a world-class tech-enabled delivery platform and software solutions across all leading technologies. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes, and is an AWS Advanced Tier Consulting Partner, a Microsoft Solutions Partner for Data & AI and Digital & App Innovation (Azure), an Oracle Partner, and a MySQL Partner.

Similar Jobs

10 Days Ago
Easy Apply
Remote or Hybrid
United States
Easy Apply
125K-234K Annually
Senior level
125K-234K Annually
Senior level
Fintech • Mobile • Software • Financial Services
Build and maintain production data pipelines, dbt models, Airflow workflows, and governed datasets supporting People Analytics. Integrate Workday and other HR systems into reliable Snowflake data products, implement data quality monitoring, row-level security, semantic models, and AI agents. Partner with engineers, analysts, data scientists, and business stakeholders to deliver trusted data solutions, communicate technical trade-offs, enforce governance, and mentor other engineers.
Top Skills: Ai AgentsAmazon RedshiftApache AirflowCi/CdDbtEtl/EltGitlabGoogle Cloud PlatformLlmsPythonSnowflakeSQLWorkday
Yesterday
Easy Apply
In-Office or Remote
IN, USA
Easy Apply
165K-227K Annually
Senior level
165K-227K Annually
Senior level
Healthtech • Information Technology • Mobile • Productivity • Software • Analytics • Telehealth
Build and operate a secure data platform supporting analysts, data engineers, machine learning engineers, and clients. Responsibilities include data isolation, access controls, auditability, MCP-based agentic workflows, pipeline and Python library reliability, and platform-level technical support. The role requires deep Python, SQL, data engineering, database, testing, and software engineering expertise in regulated environments involving sensitive data.
Top Skills: Apache AirflowAWSCortexDebeziumDockerGdprGitHipaaKafkaKubernetesMcpPythonSnowflakeSnowpark Container ServicesSoc 2SQL
2 Days Ago
In-Office or Remote
92K-164K Annually
Senior level
92K-164K Annually
Senior level
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Lead migration of on-premises ETL and database workloads to Azure and Snowflake. Design scalable data platforms using ADF, Databricks, ADLS, Logic Apps, and Python; modernize SSIS and T-SQL solutions; and develop AI-powered applications using LLMs, semantic search, vector databases, and RAG architectures. Collaborate independently across teams, optimize data operations, and support governance, quality, and advanced analytics initiatives.
Top Skills: Anthropic ClaudeAWSAzure Ai SearchAzure Ai StudioAzure Blob StorageAzure CloudAzure Data FactoryAzure Data Lake Storage Gen2Azure Machine LearningAzure OpenaiDatabricksDocument IntelligenceGoogle Cloud PlatformLarge Language ModelsLogic AppsMlflowMlopsOpenaiPrompt EngineeringPythonRetrieval-Augmented GenerationSelf-Hosted Integration RuntimeSemantic SearchSnowflakeSparkSsisT-SqlVector Databases

What you need to know about the Austin Tech Scene

Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.

Key Facts About Austin Tech

  • Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
  • Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
  • Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
  • Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
  • Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account