Sr Data Engineer
Why project44?
Our mission is to usher in a new era of trust and predictability to transportation and logistics, but there is still a lot of work to be done. The industry is massive, as is the opportunity. We are looking for bright, ambitious individuals to join our growing global team and help us enable a more productive and successful world. We are changing the way the world ships, and we are looking for you to help us get there.
About the role
The Data, Product Operations & Analytics team is a key part of project44's cross-functional product development teams, helping to drive every stage of product development through data analytic, statistical, and operational expertise.
As a Data Engineer, you'll have opportunities to work on ETL pipeline, support & solve various data integration challenges and improve core data infrastructure. This is a newly created role to bring the best of our Product & Engineering capabilities to deliver reporting and analytics solutions for our internal stakeholders.
What you’ll be doing
- Design, build and improve our ETL platform and pipelines utilizing Airflow/Argo Workflow or 3rd party tools like Looker and integrates with Data lake (Snowflake)
- Ensure standards for engineering excellence, scalability, reliability, and reusability
- Debug production issues across services and multiple levels of the stack
- Partner with the insights and data science teams to automate processes to improve data sets for analytical and reporting needs
- Write test cases, code coverage, perform QA and participate with stakeholders on UAT
You could be a great fit if you have
- At least 2+ years experience with building data pipelines using 3rd party tools (e. g., Looker) or Airflow, Spark, and/or Kafka Streams
- Good proficiency in Python, Java, or other similar languages
- Familiarity with the architecture of event collection pipelines and analytical data stores such as Snowflake
- Bachelor's Degree in computer science or equivalent experience
Technical Skills:
- Strong programming/scripting knowledge in building and maintaining ETL using SQL, Python.
- In-depth hands-on knowledge of public clouds - GCP(preferred)/AWS, PostgreSQL (version 9.6+), Snowflake, or other relational or non relational databases
- Participate in an on-call rotation to mitigate any data pipeline failures
- Experience working with APIs (Extracting data)
- Develop and deploy CICD pipelines for Data Engineering
Good to have:
- Familiar with Docker
- Experience and knowledge of optimizing database performance and capacity utilization to provide high availability and redundancy
- Proficiency with high volume OLTP Databases and large data warehouse environments
- Ability to work in a fast-paced, rapidly changing environment
- Understanding of Agile and its implementation for Data Warehouse Development
Professional Skills/Competency
- Focuses on development/improvement of framework to support repeatable and scalable solutions
- Demonstrates excellent communication and interpersonal skills; able to communicate clearly and concisely
- Takes initiative to recommend/develop innovative approaches to getting things done
- Familiarity with financial systems data
- Is a team player and encourages collaboration
- You will assume on-call responsibilities for systems that are built and owned by your team. If local regulations require – we will compensate you for the duration of on-call
- You will invest in tools, monitoring and automation to improve on-call experience for everyone on your team
Diversity & Inclusion
At project44, we're designing the future of how the world moves and is connected through trade and global supply chains. As we work to deliver a truly world-class product and experience, we are also intentionally building teams that reflect the unique communities we serve. We’re focused on creating a company where all team members can bring their authentic selves to work every day.
We’re building a company that every one of us at project44 is proud to work for, and our journey of becoming a more diverse, equitable and inclusive organization, where all have a sense of belonging, is shaped through the actions of our leadership, global teams and individual team members. We are resolute in our belief that each team member has an equal responsibility to mold and uphold our culture.
project44 is an equal opportunity employer seeking to enrich our work environment by creating opportunities for individuals of all backgrounds and experiences to thrive. If you share our values and our passion for helping the way the world moves, we’d love to review your application!
For any needed accommodations during the hiring process, please email [email protected]. Even if you don’t meet 100% of the above job description you should still seriously consider applying. Studies show that you can still be considered for a role if you meet just 50% of the role’s requirements.
More about project44
Since 2014, project44 has been transforming the way one of the largest, most important global industries does business. As transportation and logistics continue to evolve and customer expectations around delivery become more demanding, industry technology must rise to the occasion. In just a few short years, we have created a digital infrastructure that eliminates the inefficiencies caused by dated technology and manual processes. Our Advanced Visibility Platform is used by the world’s leading brands to track shipments, collaborate with supply chain partners, drive operational efficiencies, and create outstanding customer experiences.