New Relic Uncovers the Secrets Behind Google’s Fast Cyber Monday Performance

by New Relic
December 9, 2015

By Al Sargent

The holiday shopping season is a critical time for retailers. This year has already been eventful, with outages reported at Neiman MarcusTargetTime Warner CableSony PlayStationPayPal, and Royal Bank of Scotland.

As a Software-as-a-Service company ourselves, we appreciate the challenges of keeping things running 24/7. So this year we thought we’d provide some insight into the Black Friday and Cyber Monday performance of two leading e-commerce outfits: Google and Apple.

What we looked at

As we did in 2014, we ran test scripts on the subject sites in New Relic Synthetics. These scripts simulated the actions of a human user in ordering a pair of popular gifts:

The scripts ran once a minute from three different locations in data centers near Portland, San Francisco, and Washington, D.C., executing more than 7,000 times from Black Friday through Cyber Monday (November 27-30). For reference, we’ve posted the scripts on GitHub. They stopped just short of actually buying the products, so we won’t have thousands of devices to return.

Our data may indicate that even these Web giants still haven’t fully optimized the buying process during one of the year’s biggest buying periods. At this scale, even small increases in page load times could cost them literally millions of dollars in lost sales. In addition, we believe that simply looking at high-level response times isn’t enough to help understand how to optimize Web performance—we think you need to drill into the data to determine the most important improvements to make.

Super sliceable

The scripts yielded a tremendous amount of data: for every Web asset of every page, they captured the following metrics:

  • Overall load time for the URL and all dependent URLs
  • Time to load and parse the initial HTML document
  • Breakdown of load-time activities, including connection, SSL negotiation, DNS resolution, request send, block time, wait time, and receive time
  • Request body and header sizes

We looked at curated views of our performance data in Synthetics, and were easily able to create custom NRQL queries in New Relic Insights to drill down further.

Overall speediness: Google wins

The first thing we noticed was how Google was consistently fast during the entire period. Its average response time on the phone purchase script was around five seconds:

Google’s overall load time to buy a phone was consistent at around 5 seconds…

Apple, in comparison, had average response times more than twice as long, and displayed some noticeable upticks in load times on Black Friday and Cyber Monday:

… while Apple’s was more than twice as long—with noticeable spikes

Not surprisingly, both sites had some outliers that experienced longer response times. But Google’s longest times, 34 seconds, were less extreme than Apple’s, which topped out at over a minute.

Google’s worst results were slow…

… but Apple’s worst results took about twice as long

The elements of speed

Digging into the network timing components surfaced some interesting trends. Overall network timings for Google averaged less than 60 ms, and contained several milliseconds of SSL negotiation. Even while some other sites were buckling under the strain of Black Friday, Google actually got faster, dropping its average Block time from less than 20 seconds to less than 10 seconds.

Google: longer SSL negotiations but still faster overall

Apple, in contrast, had average network timings of around 80 ms, but had faster SSL negotiation times. We also saw some spikiness in Apple’s Receive times on Black Friday and Cyber Monday, although other timing elements remained steady.

Apple: slower network timings, more spikiness

When we look at resource load time, Google came out ahead again, even though it took longer to download CSS and to download fonts (something that Apple doesn’t do).

Google: faster resource load times, though font downloads added to the overall total

Apple took around 800 ms to load resources, with almost three times as long spent loading JavaScript (150 ms) to Google’s 60 ms. Apple’s biggest offender in terms of resource load time was a large graphic of a video game on an iPhone screen, a .png file that had load times as high as 9 seconds.

Apple: longer resource load times, especially for JavaScript

(Third) party time!

One surprise is how much faster Apple was in terms of time spent by third parties. Here we see that Apple had a total average third-party time of 200 ms, split across just three domains, apple.com, cdn-apple.com, and optimizely.com. Interestingly, Optimizely consumes about a third of this time, 70 ms, which shows the performance cost of Optimizely’s A/B testing and personalization services.

Apple: fewer third parties, but a significant delay for A/B testing

Google, in contrast, spent roughly four times as much time accessing third-party URLs from nine different domains. The biggest chunk of that time, around 150 ms, was consumed by fonts.googleapis.com. That begs the question, since numerous studies show that users are more likely to buy from faster websites, why not go with standard fonts? There could be a real cost to Google due to this design decision, which we detail below.

Google: many third parties, including font downloads

Single(page)-minded focus

So, why is Google faster than Apple? Synthetics’ results timeline provides some insights. Looking at results for two long Cyber Monday results for both companies, we see three things that Google is doing well:

  1. Less data. Google is transferring about 75% less data to the browser—3 MB versus 12 MB for Apple.
  2. Fewer requests. Google is making one third as many requests—114 requests versus 309 for Apple.
  3. Fewer pages. Google built its purchase process as a single-page app. Apple requires four independent page loads, each of which takes additional time. Both companies’ page loads are highlighted in red below:

Apple: 4 page loads

 

Google: just 1 page load

Conclusions

Both Apple and Google enjoyed solid performance from Black Friday through Cyber Monday, with no major outages, but Google delivered faster overall performance. Performance-oriented Web teams should consider adopting best practices from both companies, namely:

  • Stick to single-page applications to reduce total page load time.
  • Use lighter-weight pages, both in terms of bytes transferred and requests made.
  • Avoid A/B testing during major sales periods; save your A/B testing for days with lower sales volume.
  • Don’t download fonts, since they add to download times.

Throughout all this optimization, it’s important to remember the end goal: improving business outcomes. In particular, every 100 ms in increased page load times reduces sales conversions by 1%, according to an Amazon study. This argues for aggressive performance optimization across Web applications. For example, merely eliminating A/B testing could have improved Apple’s iPhone 6s sales conversions by 1%. Similar principles apply to Google’s 150 ms font download.

We welcome software-powered businesses to try out New Relic Synthetics. It runs in our Software Analytics Cloud, which provides scalability to deliver the detailed metrics required to optimize conversion rates and enable your website to deliver on its business potential.

Neha Duggal contributed to this post.

About the Author

Al Sargent is senior director of product marketing at New Relic. He's proud to lead an outstanding team of product marketing professionals who are responsible for go-to-market activities for New Relic's software analytics suite. View posts by Al Sargent.

 

Jobs at New Relic

Austin startup guides

LOCAL GUIDE
Best Companies to Work for in Austin
LOCAL GUIDE
Coolest Tech Offices in Austin
LOCAL GUIDE
Best Perks at Austin Tech Companies
LOCAL GUIDE
Women in Austin Tech