Perficient Logo

Perficient

Senior Data Engineer - Databricks

Posted 3 Days Ago
Be an Early Applicant
In-Office or Remote
Hiring Remotely in Chile
Senior level
In-Office or Remote
Hiring Remotely in Chile
Senior level
Designs, builds, and maintains large-scale data pipelines using Databricks, PySpark, and Apache Spark. Develops ETL/ELT processes, lakehouse architectures, and optimized data workflows while ensuring quality, reliability, performance, and observability. Collaborates with architects, analysts, and stakeholders to deliver scalable data solutions. Uses orchestration, version control, CI/CD, and automation practices to support modern analytics platforms.
The summary above was generated by AI
    • Design, build, and maintain end-to-end data pipelines for ingestion, transformation, and delivery of large-scale data.
    • Develop and optimize data processing logic using PySpark on Databricks (Apache Spark).
    • Implement ETL/ELT pipelines integrating data from multiple structured and semi-structured sources.
    • Contribute to the design and implementation of lakehouse architectures (Delta Lake, Medallion architecture).
    • Ensure data quality, reliability, performance, and observability across pipelines.
    • Optimize Spark jobs through partitioning, caching, and performance tuning techniques.
    • Collaborate with data architects, analysts, and business stakeholders to translate requirements into scalable data solutions.
    • Implement best practices in CI/CD, version control, and pipeline automation.
    • Support the evolution of modern data platforms and analytics capabilities.
    • Work with orchestration tools (ADF, Airflow, or similar) to manage workflows.
    • Communicate technical solutions and trade-offs effectively to both technical and non-technical stakeholders.
 

 

 

Qualifications
      • BS in Computer Science, Engineering, or related technical field, or equivalent practical experience.
      • 4+ years of experience in Data Engineering or similar roles.
      • Strong experience with Databricks and PySpark for large-scale data processing.
      • Solid knowledge of Spark architecture and performance optimization.
      • Strong proficiency in Java, Python and SQL.
      • Experience designing and building data pipelines and ETL/ELT processes.
      • Understanding of modern data architectures (lakehouse, distributed systems, big data processing).
      • Experience with version control (Git) and CI/CD practices.
      • Familiarity with cloud platforms (Azure preferred, AWS also acceptable).
      • Experience with orchestration tools (Azure Data Factory, Airflow, Fabric).
      • Strong communication skills and ability to work with cross-functional teams.
      • Advanced English (written and spoken).
      • Proactive, analytical mindset and comfortable working in fast-paced environments.

Perficient full-time employees receive complete and competitive benefits. We offer a collaborative work environment, competitive compensation, generous work/life opportunities and an outstanding benefits package that includes paid time off plus holidays. In addition, all colleagues are eligible for a number of rewards and recognition programs including billable bonus opportunities. Encouraging a healthy work/life balance and providing our colleagues great benefits are just part of what makes Perficient a great place to work. 

 

WHO WE ARE

Perficient is a leading global digital consultancy. We imagine, create, engineer, and run digital transformation solutions that help our clients exceed customers’ expectations, outpace competition, and grow their business. With unparalleled strategy, creative, and technology capabilities, our colleagues bring big thinking and innovative ideas, along with a practical approach to help our clients – the world’s largest enterprises and biggest brands succeed.

 

WHAT WE BELIEVE

At Perficient, we promise to challenge, champion, and celebrate our people. You will experience a unique and collaborative culture that values every voice. Join our team, and you’ll become part of something truly special.

 

We believe in developing a workforce that is as diverse and inclusive as the clients we work with. We’re committed to actively listening, learning, and acting to further advance our organization, our communities, and our future leaders… and we’re not done yet.

 

Perficient, Inc. proudly provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, gender, sexual orientation, national origin, age, disability, genetic information, marital status, amnesty, or status as a protected veteran in accordance with applicable federal, state and local laws. Perficient, Inc. complies with applicable state and local laws governing non-discrimination in employment in every location in which the company has facilities. This policy applies to all terms and conditions of employment, including, but not limited to, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training. Perficient, Inc. expressly prohibits any form of unlawful employee harassment based on race, color, religion, gender, sexual orientation, national origin, age, genetic information, disability, or covered veterans. Improper interference with the ability of Perficient, Inc. employees to perform their expected job duties is absolutely not tolerated.

 

Disability Accommodations:

 

Perficient is committed to providing a barrier-free employment process with reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or accommodation due to a disability, please contact us.

 

 

Disclaimer:  The above statements are not intended to be a complete statement of job content, rather to act as a guide to the essential functions performed by the employee assigned to this classification.  Management retains the discretion to add or change the duties of the position at any time. 

 

About UsPerficient is the global AI and technology consulting firm disrupting the traditional consulting model. Powered by our 7,000+ advisors, engineers, and designers, Perficient implements AI-first solutions that break conventions and deliver outcomes that matter. Proudly serving clients that represent the world’s most innovative brands, and in collaboration with our powerful technology partner ecosystem, we bring deep industry expertise and data-driven design to redefine how businesses run and succeed. Perficient is different. For real. Learn more at perficient.com.

Similar Jobs

8 Days Ago
In-Office or Remote
Senior level
Senior level
Information Technology
Design, build, and optimize large-scale data pipelines using Databricks, PySpark, ETL/ELT, and lakehouse architectures. Ensure data quality, reliability, performance, and observability while applying Spark optimization techniques. Collaborate with architects, analysts, and stakeholders to deliver scalable data solutions. Implement CI/CD, version control, workflow orchestration, and automation using tools such as Azure Data Factory, Airflow, or Fabric. Communicate technical solutions effectively across technical and non-technical teams.
Top Skills: Apache AirflowSparkAWSAzureAzure Data FactoryCi/CdDatabricksDelta LakeGitMicrosoft FabricPysparkPythonSQL
One Month Ago
In-Office or Remote
Senior level
Senior level
Information Technology
Design and maintain data pipelines, optimize data processing with PySpark on Databricks, ensure data quality, and collaborate on data architecture solutions.
Top Skills: AirflowSparkAWSAzureAzure Data FactoryDatabricksGitPysparkPythonSQL
Yesterday
Remote or Hybrid
Entry level
Entry level
Financial Services
Early-career program for recent graduates in Chile, offering structured training, technical and soft-skill development, and exposure to JPMorganChase culture. Participants work full-time in person under an initial six-month contract that may be renewed, for up to 18 months. Candidates must demonstrate advanced English, analytical and data skills, basic finance and accounting knowledge, collaboration, organization, initiative, and proficiency with AI tools. HireVue video interviews are required.
Top Skills: Generative AiHirevue

What you need to know about the Austin Tech Scene

Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.

Key Facts About Austin Tech

  • Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
  • Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
  • Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
  • Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
  • Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account