Senior Data Engineer
You're looking for a role where you can drive decisions, be challenged and build a powerful and novel product. You want to collaborate with and mentor smart, creative, and energetic people. You’re comfortable with being uncomfortable and asking for help or finding your own answers when required. You look for projects where you can learn new technologies and techniques. You thrive in an environment based on autonomy and trust. If this sounds like you, you sound like someone we want on our team.
New Knowledge is a fast-growing technology company on a mission to defend public discourse. We use AI and machine learning to protect brands from disinformation attacks and are revolutionizing the way communities, companies, and industries respond to the spread of disinformation. We are led by successful entrepreneurs, NSA veterans and disinformation industry experts. Current clients include Fortune 100 customers across a number of industries and our investors include Moonshots Capital, GGV Capital and Capital Factory.
We are looking for Senior Data Engineers with 5+ years of experience as a professional software developer, 3 years or more of which have been in data engineering/ETL. This role will be based out of our Austin office.
What You’ll Do:
- Design, develop and maintain New Knowledge’s data pipeline
- Support and monitor pipeline performance in production
- Take ownership of components of the data pipeline
- Work with product managers and data scientists to understand upcoming work and design a system capable of meeting long-term product vision
- Create and maintain documentation capable of describing how the pipeline works to a non-technical audience
- Work as part of a team to integrate new services into New Knowledge’s data pipeline
What You Have:
Note: These are not hard requirements or an exhaustive list. Consider this an outline to give you a better understanding of what you’ll be doing. We expect a senior engineer to meet at least half of these requirements and be comfortable and excited to learn the rest.
- Experience owning pieces or all of a data pipeline working as an ETL or data engineer
- Experience working on data driven applications with data science and/or application engineers
- Experience processing social media, text and image data is a plus
- Experience or familiarity with Kafka or similar distributed systems (knowledge of schema registry/data types and serialization options and partition strategies a plus)
- Experience with Docker and Kubernetes
- Experience or familiarity with popular stream processing framework such as Spark Streaming, Kafka Streams, Flink or similar
- Experience supporting large scale batch analytics in Hadoop ecosystem (loading and retrieving data)
- Experience working with or building schedulers, workflow automation/coordination tools
- Experience implementing tests and sanity checks on large complex data pipelines
- Experience helping other developers write performant SQL queries
- You enjoy working as part of a team and mentoring junior developers
- You have strong written communication / documentation
- You have a desire to code
- You are not afraid to tackle problems you’ve never encountered before
What You’ll Get:
- Chance to do purpose-driven work with smart, enthusiastic people
- Competitive salary and healthcare
- 401(k) with a 4% company match
- A leadership team that trusts and embraces technology
- Flexible work hours
- Unlimited vacation that you’re encouraged to take
- Stocked kitchen full of healthy snacks and Topo Chico
- Annual stipend to purchase the gadgets, books, subscriptions, and other tools you need
- Weekly catered lunches
- Monthly team-wide events like Alamo Drafthouse movie screenings, escape rooms, and stand up paddleboarding
- Many more benefits … apply to find out!