Lead data engineering teams to design, build, and maintain scalable cloud-native data pipelines, data platforms, and data products using Snowflake, Databricks, Airflow, and modern architectures. Optimize ETL/ELT, SQL, and transformations for analytics, reporting, and AI use cases; ensure data quality, governance, observability, and stakeholder alignment. Produce documentation, runbooks, and present technical solutions to business and technical audiences.
EXL is seeking a Senior Data Engineer to join our Data & Analytics practice and support strategic client engagements. This role will be responsible for designing, building, and managing scalable cloud-based data platforms, driving modern data engineering practices, and leading teams delivering high-impact data solutions. The ideal candidate will combine strong technical expertise with consulting and leadership capabilities, partnering closely with client stakeholders to translate business requirements into scalable data architectures and engineering solutions.
Data Engineering & Platform Development
- Design, develop, and maintain scalable data pipelines and data products supporting analytics, reporting, AI, and operational use cases.
- Build and optimize ETL/ELT frameworks for large-scale data ingestion, transformation, validation, and consumption.
- Develop and manage cloud-native data platforms leveraging Snowflake, AWS, Apache Airflow and modern data architectures.
- Create scalable data models, data marts, semantic layers, and curated datasets that support enterprise analytics initiatives.
- Optimize SQL workloads, transformation logic, and query performance to improve scalability and cost efficiency.
- Establish reusable engineering frameworks, accelerators, and best practices to improve delivery consistency across projects.
- Ensure high standards of data quality, reliability, governance, and observability throughout the data lifecycle.
- Develop and maintain Snowflake-based data ecosystems, leveraging advanced features for performance optimization and data sharing.
- Build and orchestrate data workflows using Airflow and other workflow scheduling platforms.
- Collaborate directly with client stakeholders to gather requirements, define roadmaps, and develop scalable technical solutions.
- Present solution designs, technical recommendations, and project updates to both technical and business audiences.
- Prepare and maintain comprehensive project documentation, technical specifications, architecture diagrams, and operational runbooks.
Required Qualifications
- 4+ years of experience in Data Engineering, Big Data Engineering, or Cloud Data Platform development.
- Bachelor's or Master's degree in Computer Science, Engineering, Analytics, Mathematics, Information Systems, or related disciplines.
- Strong hands-on expertise in SQL, Python, and PySpark.
- Extensive experience working with Snowflake, Databricks, or similar cloud-native data platforms.
- Proven experience building and supporting large-scale ETL/ELT data pipelines.
- Strong understanding of data warehousing concepts, dimensional modeling, and modern Lakehouse architectures.
- Experience implementing Medallion Architecture and enterprise-grade data modeling practices.
- Hands-on experience with workflow orchestration tools such as Apache Airflow or equivalent scheduling frameworks.
- Experience working with cloud ecosystems including AWS, Azure, or GCP.
- Strong knowledge of performance tuning, optimization, monitoring, and operational support for data platforms.
- Demonstrated experience leading engineering teams and coordinating with client and internal stakeholders.
- Excellent analytical, problem-solving, communication, and stakeholder management skills.
-
Ability to work independently and lead complex initiatives in fast-paced consulting environments.
Preferred Qualifications
- Experience with streaming and real-time data processing frameworks.
- Familiarity with DataOps, CI/CD, Infrastructure as Code, and DevOps practices.
- Experience with data governance, data quality frameworks, and metadata management.
- Exposure to AI/ML data pipelines and feature engineering workflows.
- Experience with visualization tools such as Tableau, Power BI, or Looker.
- Hands-on experience with Big Data technologies including Spark, Hadoop, Hive, HBase, Kafka, or related platforms.
-
Consulting or client-facing delivery experience in enterprise-scale environments.
Similar Jobs
Artificial Intelligence • Information Technology • Machine Learning • Generative AI • Big Data Analytics
Lead customer-facing data engineering engagements from architecture through production handover. Build and optimize batch and streaming Lakehouse pipelines on Databricks using PySpark, SQL, and Delta Lake. Establish Medallion architecture conventions, implement data quality and automated testing, optimize Spark performance and platform costs, and mentor engineers. Partner with architects and explain technical trade-offs to customers.
Top Skills:
AirflowSparkAWSAzureCi/CdDatabricksDatabricks Asset BundlesDbtDelta LakeGCPGitLakeflowPysparkPythonSQLUnity Catalog
Information Technology
Own the design and operation of Navanta’s data backbone, including secure bank-system ingestion, lakehouse architecture, semantic data modeling, reconciliation, quality, observability, and governance. Partner with AI/ML, security, and platform teams to deliver governed, queryable data products. Lead data contracts, lineage, PII classification, access controls, pipeline reliability, and onboarding of new bank environments.
Top Skills:
AirflowApache IcebergDagsterDbtDebeziumDuckdbKafkaLakekeeperMinioNessiePgvectorPolarisPostgresPostgresql Logical ReplicationPrestoPythonRedpandaS3SQLSQL ServerSql Server CdcTrino
Solar • Renewable Energy
Lead the rebuild and modernization of the company’s data platform by designing and implementing an Azure and Databricks lakehouse. Responsibilities include managing Azure Data Factory pipelines, developing Delta Lake architectures, creating data models and transformations with SQL and Python, establishing development standards, monitoring data quality, troubleshooting performance issues, leading technical projects, and mentoring or hiring data engineering staff.
Top Skills:
AirbyteAzureAzure Data FactoryCi/CdDatabricksDelta LakeEstuaryFivetranGitInformaticaMicrosoft DynamicsNetSuiteOraclePower BIPythonSAPSQLSsisUnity Catalog
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
