Design, build, and maintain scalable data pipelines, data marts, and schemas; collaborate with data scientists and stakeholders; deploy and maintain data storage and cloud infrastructure; implement data validation and quality checks; optimize pipeline performance and evaluate new tools to improve data engineering capabilities.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and design appropriate solutions.
- Work closely with IT and infrastructure teams to deploy and maintain data storage and processing systems, including databases, data warehouses, and cloud infrastructure.
- Design, build, and maintain robust, scalable data pipelines to ingest, process, and transform data from various sources, including internal databases and external APIs.
- Optimize data pipelines for performance, scalability, and reliability, ensuring timely and accurate delivery of data.
- Implement data validation and quality checks to identify and rectify any issues in data pipelines, ensuring data integrity.
- Explore and evaluate new technologies, tools, and frameworks to improve data engineering processes and capabilities.
- Perform data modeling and schema design to support analytical and reporting requirements.
- Design and build data marts and business layers to facilitate efficient data access and analysis.
Requirements
- Bachelor’s degree in Computer Science, Information Systems, or other related technical field or equivalent work.
- Proven experience as a Data engineer or similar role, with at least 4 years of experience.
- Advanced SQL knowledge/experience - complex queries and query optimization.
- Proficiency in programming languages such as Python.
- Experienced in the development of data warehouses and proficient in data modeling.
- Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.
- Experience in building data streams using technologies such as Kafka, Flink, or Spark Streaming is considered a plus.
- Strong problem-solving skills and attention to detail.
- The ability and willingness to learn new technologies on the job.
- Self driven, highly motivated, fast learner, ambitious and creative.
Similar Jobs
Information Technology
Design, build, and operate scalable cloud-native data pipelines and infrastructure to support analytics and AI workloads. Develop dimensional and semantic data models and curated data products, optimize cloud data services, maintain versioned development workflows, and collaborate with AI, Analytics, and Product teams to deliver high-quality data for LLMs, agents, and RAG solutions.
Top Skills:
Ai AgentsAirflowAthenaAWSDbtGitGlueLambdaLlmsPrefectPythonRagRedshiftS3SQL
Information Technology
Design, build, and maintain scalable batch and near-real-time data pipelines and models. Ensure data quality, observability, security, and cost-efficient performance. Partner with cross-functional teams to enable analytics, support BI use cases, and mentor junior engineers while applying DevOps practices and incident management.
Top Skills:
AirflowAPIsAWSAzureBigQueryCdcCi/CdDagsterDatabricksDbtDelta LakeDockerFivetranFlinkGCPGitIcebergJavaKafkaKubernetesLookerMetabasePower BIPrefectPythonRedshiftScalaSnowflakeSparkSQLStitchSupersetTableauTerraformWebhooks
Software
Lead design, development, and maintenance of end-to-end analytical solutions on Microsoft on-premises and Azure cloud. Elicit requirements, mentor junior engineers, deliver analytical and real-time solutions (ETL, data lake/warehouse, governance), and transfer knowledge to clients.
Top Skills:
Active DirectoryAzure Data CatalogAzure Data FactoryAzure Data Lake AnalyticsAzure Data Lake Storage Gen2 (Adls Gen2)Azure DevopsAzure FunctionsAzure HdinsightAzure Key VaultAzure PurviewAzure SqlAzure Stream AnalyticsAzure Synapse AnalyticsAzure Synapse Analytics Dedicated PoolsCosmosdbDatabricksHiveKafkaMs Sql ServerPolybasePower BISnowflakeSparkSsasSsisSsrsU-Sql
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
