Lead Site Reliability Engineer

Join us as we work to create a thriving ecosystem that delivers accessible, high-quality, and sustainable healthcare for all.

About Our Company:

At athenahealth, we deliver high quality and affordable healthcare solutions and drive growth across industries. Our success is powered by our talented team and the strategic leadership of our corporate managers. We foster a collaborative and dynamic environment where employees are encouraged to innovate, grow, and excel in their careers. As part of our team, you’ll be empowered to make a significant impact, lead strategic initiatives, and drive business results.

Position Overview:

We are looking for a Lead Site Reliability Engineer to join our Cloud Engineering division.  Cloud Engineering ensures the continuous availability of the technologies and systems that are the foundation of athenahealth’s services.   We are directly responsible for thousands of servers, petabytes of storage, and handling thousands of web requests per second, all while sustaining growth at a meteoric rate. We enable an operating system for the medical office that abstracts away administrative complexity, leaving doctors free to practice medicine.

But enough about us; let’s talk about you!

You’re a seasoned engineer with a passion for identifying and resolving reliability and scalability challenges.  You are a curious team player, someone who loves to explore, learn, and make things better.  You are excited to uncover inefficiencies in business processes, creative in finding ways to automate solutions, and relentless in your pursuit of greatness. You’re a nimble learner capable of quickly absorbing complex solutions and an excellent communicator who can help evangelize engineering excellence.

The Team:

We are a bunch of Site Reliability Engineers who are passionate about reliability, automation, and scalability. We use an agile based framework to execute our work, ensuring we are always focused on the most important and impactful needs of the business. We support systems in both private and public cloud and make data-driven decisions for which one best suit the needs of the business. We are relentless in automating away manual, repetitive work so we can focus on projects that help move the business forward.

Job Responsibilities:

Reliability and Availability:

• Define, measure, and maintain Service Level Objectives (SLOs) and Service Level Indicators (SLIs) for cloud services and infrastructure components.

• Lead efforts to continuously improve system availability, fault tolerance, and disaster recovery capabilities.

• Ensure proactive incident detection, efficient root cause analysis, and timely resolution of production incidents

• Participate in a 12x7 on-call rotation. We have a peer team in India that manages the overnight on-call

Automation and Infrastructure as Code (IaC):

• Drive automation efforts to reduce manual intervention and streamline cloud infrastructure management.

• Implement Infrastructure as Code (IaC) using tools like Terraform, AWS CloudFormation, and Ansible to provision, manage, and scale cloud resources.

• Automate deployment, scaling, and monitoring processes to improve efficiency and reduce operational complexity.

Monitoring, Observability, and Performance Tuning:

• Design and implement monitoring, logging, and alerting solutions to track cloud infrastructure health, performance, and security.

• Use observability tools (e.g., Prometheus, Grafana, Cloud Watch) to ensure continuous visibility into cloud infrastructure performance and capacity.

• Identify bottlenecks and performance issues, proposing and implementing improvements to ensure optimal resource usage.

Security and Compliance:

• Ensure that cloud infrastructure is built with security best practices in mind and meets all relevant compliance and regulatory requirements.

• Collaborate with security teams to implement security controls and risk mitigation strategies across cloud environments.

• Regularly audit and review cloud infrastructure for security vulnerabilities and compliance gaps.

Collaboration and Cross-Functional Leadership:

• Work closely with development, DevOps, and operations teams to ensure cloud infrastructure aligns with application and business requirements.

• Lead and mentor a team of Site Reliability Engineers, promoting best practices and fostering a culture of operational excellence.

• Act as a key technical point of contact for cloud-related infrastructure and operations issues.

Incident Management and Post-Mortem:

• Lead the incident response efforts for cloud infrastructure-related issues, ensuring that all incidents are managed effectively.

• Conduct post-incident reviews (PIRs) to identify root causes and implement preventive measures.

• Continuously refine incident management processes to reduce downtime and enhance recovery times.

Required Skills and Qualifications:

• 10 years of hands-on experience with cloud automation and configuration management tools (e.g., Terraform, AWS CloudFormation, Ansible, Puppet).  On a Hybrid Cloud Set-up.

• 7+ years of experience in a Site Reliability Engineering (SRE), Infrastructure Engineering, or DevOps role, with at least 3+ years in a technical leadership capacity.

• Deep knowledge of cloud services and technologies (e.g., EC2, S3, Lambda, Kubernetes, etc.).

• Proficiency in scripting or programming languages (Python, Go, Bash, etc.).

• Experience with monitoring, logging, and observability tools (e.g., Prometheus, Grafana, Datadog, ELK stack).

• Familiarity with Continuous Integration/Continuous Deployment (CI/CD) pipelines and cloud-native development practices.

• Strong expertise in managing cloud infrastructure (AWS, Google Cloud, Azure) in production environments.

• Experience with cloud-native architectures, microservices, and containerized environments (Kubernetes, Docker)

• Proven experience in building and managing highly available, scalable, and fault-tolerant systems in the cloud

• Strong understanding of cloud networking, storage, compute services, On-Prem and security best practices

• Strong knowledge of Linux administration and internals

• Effective communication skills, with the ability to translate technical concepts to non-technical stakeholders.

Preferred Qualifications:

• Bachelor’s degree in Computer Science, Engineering, or a related field

• Knowledge of database systems such as MySQL, Oracle or PostgreSQL

• Experience with managing on-prem infrastructure at scale

• Certifications in AWS, RedHat5 or relevant technologies are a plus

• Experience running containerized workloads (Kubernetes, Docker) in production

Expected Compensation

The base salary range shown reflects the full range for this role from minimum to maximum. At athenahealth, base pay depends on multiple factors, including job-related experience, relevant knowledge and skills, how your qualifications compare to others in similar roles, and geographical market rates.  Base pay is only one part of our competitive Total Rewards package - depending on role eligibility, we offer both short and long-term incentives by way of an annual discretionary bonus plan, variable compensation plan, and equity plans.