Lead Data Engineer at BigCommerce
Sorry, this job was removed at 6:02 a.m. (CST) on Tuesday, April 13, 2021
BigCommerce is disrupting the e-commerce industry as the SaaS leader for fast- growing, mid-market businesses. We enable our customers to build intuitive and engaging stores to support every stage of their growth.
We are looking for a Lead Data Engineer to join our Data Engineering team. Leveraging your big data skills, you will help design and build a Realtime, centralized data distribution service for our core business systems to achieve faster and bigger results. This involves deciphering value from several systems with a variety of access patterns. You will report to Sr. Director of IT and work with cross-functional teams in a highly visible role. The Senior Data Engineer will mentor junior engineers and communicate with senior leadership.
What you’ll do
- Build scalable near real time data pipelines on AWS and GCP that collects, transforms, loads and curates data from various internal and external data sources
- Build a scalable distributed data store that will be central source of truth
- Create scalable and low latency solutions
- Implement machine learning pipelines and algorithms
- Evaluate new technologies and build prototypes for continuous improvements in Data Engineering
- Contribute to setting standards and development principles for the team and influence across the larger organization
Quality Skills
- Write automated unit, integration and acceptance tests to support our continuous integration pipelines
- Build performance and load tests written from scalability and resiliency standpoint
- Participate in peer code reviews and advocate for the best coding practices and principles
- Partners with Infrastructure and Engineering teams to ensure instrumentation, logging and monitoring is in place
Team / Leadership Skills
- Actively participate with our development team in all phases of the software development lifecycle, including requirements gathering, functional and technical design, development, testing and roll-out, and support
- Estimate work items and help maintain a scrum backlog
- Lead collaboration across different teams
- Distill and present research findings to both technical and non-technical leaders
- Plays a key role in defining data engineering strategy and communicating priorities to others
- Coach and mentor others, models the company values
- Partners with engineering management to determine hiring needs and helps with recruiting activities
Who you are
- BS or MS in Computer Science or equivalent experience or field
- 8+ years developing data and software solutions
- 4+ years of experience in Java, Scala and/or Python
- 4+ years of experience in building data pipelines in the cloud with tools like S3, Hadoop, Hive; or their equivalents
- 4+ years of experience in streaming technology such as Kafka
- Strong experience with AWS and/or GCP
- Strong experience with SQL, ETL, Data Warehousing and Data Orchestration
- Experience with scheduling frameworks, preferably Airflow
- Experience with monitoring near real time scalable data pipelines
- Experience with of all aspects of data systems including database design, ETL, aggregation strategy, performance optimization, and technology trade-offs
- Experience with Terraform, Puppet, and Jenkins
- Experience with shell scripting
- Focused developer with a strong sense of ownership
- Ability to independently drive projects consisting of many stories from inception through to completion and production deployment
- Capable of working closely across multiple teams to ensure data solutions line up with business initiatives and are of high quality
- Excellent written and verbal communication skills
Diversity, Equity & Inclusion at BigCommerce
We have the opportunity to build not only a great business but a great company, with soul. Our beliefs and commitment to diversity, equity and inclusion are a central part of achieving that.
Our dedication to DEI is grounded in two things: a moral belief in the dignity, value, and potential of every individual, and a practical belief that diverse, inclusive teams will create the best outcomes for our customers, partners, employees, and company. We welcome everyone to be a part of our journey.
Current BigCommerce Employees: Please use the internal job board to apply for openings
Read Full Job Description
