As a Data Engineer, you will develop data pipelines and APIs, configure cloud resources in AWS, and translate business needs into technical specifications.
As a Data Engineer, you will be part of a talented team of engineers responsible for the deployment and configuration of cloud resources to meet individual client business needs in AWS. Client engagements cover a wide variety of business requirements and require our engineers to adapt quickly and stay on top of recent cloud technology trends. Candidates should be able to identify and remediate issues within cloud-based systems, based on their knowledge of industry standards and best practices.
Responsibilities
Responsibilities
- Work with the team to evaluate business needs and priorities, liaise with key business partners and address team needs related to data systems and management
- Translate business requirements into technical specifications; establish and define details, definitions, and requirements of applications, components and enhancements
- Participate in project planning; identifying milestones, deliverables and resource requirements; tracks activities and task execution
- Generate design, development, test plans, detailed functional specifications documents, user interface design, and process flow charts for execution of programming
- Develop data pipelines / APIs using a variety of tools, including but not limited to Python, SQL, Spark and various AWS services
- Use an analytical, data-driven approach to drive a deep understanding of fast changing business
- Build large-scale batch and real-time data pipelines with data processing frameworks in AWS
- Moving data from on-prem to cloud and cloud data conversions
- Experience in data engineering with an emphasis on data pipelines, analytics and reporting
- Exposure to the AWS Cloud Platform
- Experience in SQL, data transformations, and troubleshooting across at least one database Platform (Redshift, Amazon RDS, Cassandra, Snowflake, PostgreSQL, Databricks, etc.)
- Experience in the design and build of data extraction, transformation, and loading processes by writing custom data pipelines
- Experience in a scripting languages such as Python
- Experience designing and building solutions utilizing various Cloud services such as EC2, S3, EMR, Kinesis, RDS, Redshift/Spectrum, Lambda, Glue, Athena, API gateway, etc.
- AWS Certification - Solutions Architect Pro or Data Specialty
- Machine Learning experience
- Advanced GenAI experience
Compensation:
The expected base salary range for this position is $115,000 – $130,000 USD annually, plus eligibility for an annual performance bonus.
Actual compensation will depend on experience, skill set, and location.
Benefits:
We offer a comprehensive benefits package that includes medical, dental, and vision coverage, and paid time off.
Similar Jobs
Financial Services
Design, build, and optimize end-to-end data pipelines and architectures using Databricks, Spark, AWS, Airflow, and Kafka. Develop ETL processes, data models, warehouses, and reliable Python and SQL solutions while ensuring data quality, security, and governance. Collaborate with data scientists, analysts, and stakeholders to support analytics and AI/ML initiatives. Monitor workflows, troubleshoot issues, validate AI-assisted outputs, and improve performance and resiliency.
Top Skills:
Amazon RedshiftAmazon S3Apache AirflowApache KafkaSparkAWSAws LambdaDatabricksDockerKubernetesPythonSQL
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Architects, deploys, and operates scalable data platform infrastructure supporting analytics. Responsibilities include administering Airflow and Superset, managing Terraform infrastructure, building CI/CD pipelines, implementing security and compliance controls, developing dbt data models, cataloging assets in OpenMetadata, automating with Python, and mentoring engineers. The role requires extensive infrastructure and data engineering experience, cloud expertise, container orchestration, and eligibility for CJIS clearance.
Top Skills:
Ai TechnologiesApache AirflowApache SupersetAWSAzureCi/CdDbtDockerGCPIamKubernetesMachine Learning PipelinesOciOpenmetadataPythonSQLTerraform
Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Design and develop data pipelines, integrations, transformations, and infrastructure across AWS, Snowflake, Airflow, and MySQL. Troubleshoot warehouse issues, validate data quality, improve pipeline reliability, and translate business needs into data engineering solutions. Interpret technical findings for client recommendations, manage workstreams, review junior team members’ work, and provide technical guidance across managed services engagements.
Top Skills:
Apache AirflowAWSCloud PlatformsData LakesData WarehousesMySQLSnowflake
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
