Design, build, and maintain AWS-based data pipelines and ETL processes using Glue, Spark/PySpark and EMR. Integrate multiple data sources into data lakes and Redshift, implement DevOps CI/CD practices, and work with Kafka, Hudi, and various AWS storage and analytics services.
This is a remote position.
- 3+ Years of Development / Data Engineering Experience
- 2+ Years Developing with AWS Cloud solutions
- 2+ Years Developing in Spark/PySpark
- 2+ Years Developing AWS Glue ETL
- 2 + Years AWS Storage models (S3 and DynamoDB)
Some hands-on developing with on-prem ETL tooling (Ab Initio preferred, Informatica)
Requirements
- Bachelor of degree in computer or any related field.
- Minimum of 12+ years’ experience in the said role.
- Strong experience in AWS GLUE, EMR, Hudi, experience extracting data from multiple sources & loading data into Data Lakes, AWS Redshift
- Experience working with AWS Elastic search, RDS, PostgreSQL preferred
- Experience w/ AWS services such as Lambda, EMR, SNS/SQS, Event Bridge, Lake Formation & Athena
- Experience w/ integrating applications/systems (data producers) with Enterprise Kafka topics (Confluent Kafka integration w/ AWS S3, Redshift)
- Experience in Requirements Analysis, Data Analysis, Application Design, Application Development & Integration Testing
- Working knowledge of on-prem Extraction, Transformation, Cleansing and Loading methodology and principles
- Experience implementing and adding to DevOps principles (GitLab, maintaining CI/CD pipelines)
- Experience in Java svcs is nice to have
N/A:- Please kindly note that this role is for USC and GC Visa holders only.
Interested and qualified candidates should send their resume to [email protected] using the job title as the subject of the mail.
Similar Jobs
Information Technology • Software
Own and scale AWS OpenSearch production infrastructure and PostgreSQL-to-OpenSearch data pipelines. Design indices and mappings, optimize queries and bulk indexing, monitor cluster health, implement ingestion reliability, perform reindexing/backfills, build reconciliation and recovery processes, and collaborate with backend, DB, and product teams on searchable schemas and runbooks.
Top Skills:
Amazon S3Amazon SqsAws DmsAws EcsAws EksAws LambdaAws Opensearch ServiceClickhouseCloudwatchElasticsearchGoIamJavaOpensearchOpensearch IngestionOpensearch Query DslPostgresPythonTerraformTinybird
Information Technology • Software
Own and scale AWS OpenSearch infrastructure and PostgreSQL-to-OpenSearch pipelines. Design indices, mappings, analyzers, and retention policies. Optimize queries and bulk indexing, monitor cluster health, implement retries/idempotency, backfills, reconciliation, and disaster recovery. Build dashboards, alerts, runbooks, and participate in incident response while collaborating with backend, DB, infra, and product teams.
Top Skills:
Aws DmsAws LambdaAws Opensearch ServiceClickhouseCloudwatchEcsEksElasticsearchGoIamJavaJvmOpensearchOpensearch IngestionOpensearch Query DslPostgresPythonS3SqsTerraformTinybird
Gaming • Software
Design, build, and maintain scalable AWS-based data pipelines and storage (S3, Redshift, RDS, DynamoDB). Implement IaC (CloudFormation/CDK), ETL optimization, monitoring, security (IAM, OWASP/CIS best practices), and CI/CD/code pipeline controls. Collaborate with product teams, document solutions, and provide training to support analytics and reliable data infrastructure.
Top Skills:
Amazon DynamodbAmazon KinesisAmazon RdsAmazon RedshiftAmazon S3AWSAws CdkAws CloudformationAws Data PipelineAws GlueAws IamCis Aws Foundations BenchmarkCweETLInfrastructure-As-CodeMonitoring And LoggingOwaspStatic Code Analysis
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
