Build and optimize scalable data pipelines and cloud-based data solutions using Python, PySpark, SQL, and Snowflake. Integrate REST APIs and external data sources, develop ETL/ELT workflows, optimize Snowflake queries, and implement data quality, validation, monitoring, and automation. Collaborate with engineering and analytics teams to deliver reliable data platforms.
Job Description
Data Engineer - Python, PySpark, SQL, Snowflake & API Integration
Job Description:
We are seeking a skilled Data Engineer with strong hands-on experience in Python, PySpark, SQL, Snowflake, and API integrations to build scalable data pipelines and cloud-based data solutions.
Key Responsibilities:
- Design, develop, and optimize data pipelines using Python, PySpark, SQL, and Snowflake.
- Integrate REST APIs and external data sources into Snowflake-based data platforms.
- Build and maintain ETL/ELT workflows for data ingestion, transformation, and processing.
- Develop efficient SQL queries and optimize Snowflake performance.
- Implement data quality, validation, monitoring, and automation.
- Collaborate with engineering and analytics teams to deliver reliable data solutions.
Required Qualifications:
- 3+ years of experience in Data Engineering.
- Strong hands-on expertise in Python, PySpark, SQL, and Snowflake.
- Experience with REST APIs, JSON, and API-based data integration.
- Knowledge of ETL/ELT processes, data modeling, and data warehousing.
- Experience with cloud platforms such as AWS, Azure, or GCP is preferred.
Additional Information
All your information will be kept confidential according to EEO guidelines.
Similar Jobs
eCommerce • Healthtech • Kids + Family • Retail • Social Media
Own and evolve Babylist's AWS infrastructure, Terraform IaC, Kubernetes/EKS clusters, CI/CD, and observability for a platform serving millions. Lead incident response, improve developer tooling, and set reliability standards across engineering teams.
Top Skills:
AWSCdnCircleCICloud NetworkingCronitorDatadogDnsEksGithub ActionsKubernetesLoad BalancersMySQLPagerdutyRdsRedisRuby On RailsSentrySidekiqTerraform
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Owns Pfizer GH&B’s AI-ready Snowflake data warehouse, data architecture, ontologies, governance, quality, lineage, access controls, and production reliability. The role supports AI/ML models, dashboards, and business-critical data applications; establishes testing, release, monitoring, and incident processes; directs contingent technical resources; and aligns business-specific data decisions with enterprise standards. Requires strong SQL, Python, Collibra, data warehousing, unstructured-data architecture, ontology, governance, and stakeholder-influence experience.
Top Skills:
Ai/MlAlationCollibraGraph DatabasesMicrosoft PurviewProtégéPythonRdf/OwlSnowflakeSQL
Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
Lead the AI Quality engineering team while remaining hands-on building evaluation infrastructure, observability tooling, diagnostic systems, datasets, annotation workflows, and CI/CD pipelines for production AI agents. Run experiments across prompts, models, routing, caching, and agent harnesses to improve quality, latency, and cost. Partner with AI Core engineering and manage technical direction and team priorities.
Top Skills:
Ai AgentsBraintrustCi/CdLangsmithLlmsPythonRuby On Rails
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
