Senior Data Engineer

Data is at the core of Compass. Our organization is responsible for evaluating, accelerating, building, and maintaining a unified, scalable, and cost-effective data ecosystem—including infrastructure, data lakes, data warehouses, and tooling. As a horizontal team, we manage data across various domains such as Agents, Listings, Segments, Finance, and more.

We are seeking a self-driven Senior Data Engineer to lead key initiatives and maximize the value of our data. In this role, you will collaborate with stakeholders across Analytics, Growth, Finance, Product, and Data Science teams to understand business needs and design optimal solutions. The ideal candidate is an experienced data expert who can build and optimize data systems for BI, insights, product development, and AI/ML applications.

We are looking for individuals who excel at solving ambiguous business challenges through data, take ownership of their work, and proactively communicate progress and blockers to leadership. If you thrive in a dynamic environment and are passionate about driving impact through data, we’d love to hear from you!

Responsibilities

• Design, develop, and operationalize scalable and reliable data pipelines.
• Build and manage large, complex data sets that align with both functional and non-functional business requirements.
• Apply expertise in database design, including logical data modeling, physical optimization, and security best practices.
• Enhance data pipeline integrity by implementing Data Quality checks, ensuring a comprehensive and reliable data service.
• Collaborate with Analytics and Data Science teams to uncover insights and hidden patterns, driving value-added data solutions.
• Provide on-call support, troubleshooting and resolving user issues based on severity to ensure smooth data operations.

Qualification

• 7+ years of experience in designing, engineering, and operationalizing data pipelines for large and complex datasets.
• 3+ years of hands-on experience working with unstructured data, including JSON.
• Strong expertise in batch and stream processing methodologies.
• Extensive proficiency in Advanced SQL for data transformation and analysis.
• 3+ years of experience in Python/PySpark for data processing and automation.
• 3+ years of experience in data transformation using Databricks.
• 3+ years of experience working with AWS cloud infrastructure or any equivalent cloud environment.
• 2+ years of experience in orchestrating data pipelines using DBX DLT, Airflow, or similar tools.
• Proficient in GIT (CI/CD) for version control and deployment automation.
• Bonus Skills: Experience with Power BI for data visualization and Graph Databases for advanced data modeling.

Compensation: The base pay range for this position is $140k-$170k annually; however, base pay offered may vary depending on job-related knowledge, skills, and experience. Bonuses and restricted stock units may be provided as part of the compensation package, in addition to a full range of benefits. Base pay is based on market location. Minimum wage for the position will always be met