Measuring How Speed Impacts Visitors
Last Updated: April 13, 2020
When it comes to measuring a website’s speed, there are the standard go-to tools: Google Page Speed Insights, GT Metrix, Web Page Test, Pingdom, and similar. However important these tools are—and they are important—they only look at one aspect of speed: load time.
These are incredibly helpful metrics. And, before we continue, if you aren’t using these tools, please begin using these tools to understand speed on your website. You can use my free guide to begin learning about measuring load times, including using these tools.
However, while incredibly important, these tools aren’t the full story. What these tools don’t tell you as clearly is the extent to which speed is affecting your website’s visitors. And, without knowing that, you can’t really know if your website speed is fast enough or needs improvement.
When Load Time Does & Doesn’t Matter
What? How can I say that? If your website takes five seconds to load on desktop devices, then that is obviously affecting your website’s visitors and can only mean lower conversions, lower engagement, lower rankings.
That is often the case.
But not always.
Those traditional tools may indicate it takes 5+ seconds for a website to load on desktop devices but that website still might drive lots of conversions or earn high rankings. Yes, improving the speed might help the website, but the speed isn’t exactly harming the website given its high conversion rate and rankings.
Why? Well, a website’s load time, while important for CRO and SEO, isn’t the only factor at play. In some (likely most) cases, speed may not even be the most important factor. Perhaps a strong brand reputation, an otherwise-great user experience, great prices on products, or great content more than offsets the sluggish speeds.
Similarly, sometimes you can have fast load times when checking a website’s speed in the traditional speed tools but still have a speed problem affecting engagement, conversions, or rankings. As one example, think about a website that loads almost all its content dynamically via client-side code (with something like React). That type of website will almost certainly load quickly as measured by the standard speed tools. However, search engines robots will often struggle to see content loaded using these methods and, if the robots can’t see the website’s content, the content won’t appear in rankings.
In short: it is important to use those standard tools but, equally, it is important to not use those tools in isolation. In order to gain a deeper understanding of speed on your website, you also want to measure the way speed impacts your visitors. Let’s talk about methods you can take to measure the way speed affects visitors, including how some of these methods can help you get a fuller understanding of speed on your website.
One way, and I’d argue one of the best ways, to determine the value of speed optimizations is with correlations. After you’ve run your speed tests and you know your website’s load times across a variety of devices and connection speeds, you want to compare those speeds to the page’s performance metrics. Do faster pages have lower bounce rates? Do slower pages rank in lower positions? Are conversions concentrated around pages that start rendering content in under one second?
Let’s say an ecommerce website has five product pages that load quickly, ten product pages that load somewhat slower but still load at decent speeds for most devices, and another fifteen product pages that load quite slow on several different devices. Before the company behind this website invests time or money improving speed on those ten moderately slow pages or the fifteen slow pages, the question to figure out is if those five fast pages perform better—do those pages have higher rankings, better conversion rates, or better engagement rates? And, critically, can those differences in a page’s performance be explained in any other way?
If those correlations show that there is a measurable difference in conversions or rankings for pages that load faster, and that no other factors can explain that difference, then you can know that improving the speed is worthwhile. However, in doing this correlation, you might find that those five pages that happen to load quickly might perform better because they happen to be popular products with customer loyalty or maybe those are products that have less competition so are able to rank better in search results. Depending on what you find from the correlations, that changes whether you need to invest in improving speed or not.
A word of caution is that this type of measurement is most helpful when you’ve reached an acceptable baseline on speed. If your website takes twenty or thirty seconds to load, common sense and best practices would tell you that improving speed is a worthwhile investment. Even if you somehow manage to have a decent conversion or engagement rate with such a long load time, those will almost certainly improve once the load time is reduced. However, once your website’s load times are within reasonable levels (under 5-8 seconds on most devices), you need to know if more work on speed could help and be worth the investment relative to everything else you could invest in improving.
Perception Test: How Fast Does It Feel?
Reviewing speed metrics only gives you an understanding of how fast the website loads but these metrics tell you nothing about how people feel about the website’s load. Even with correlating speed to engagement or conversion rates, you still can’t get a direct sense of what people are thinking. Are people satisfied with your website’s load time or are they annoyed at your website? Even when annoyed, they still might (begrudgingly) engage and convert.
The best way to measure the perception of speed is as part of usability testing. After people have used the website, completing whatever tasks you asked them to complete during the usability test, you can then ask how they felt about the website’s speed. You want to know if that person felt they website should have loaded faster or if the speed was acceptable as is. Truth is, most people will volunteer their opinions about a website’s speed during a usability test even without prompting—especially if the website is loading slowly.
This information gives you direct insight into what people are thinking. However, people may give inaccurate answers—indicating the website’s speed was fine, even though it wasn’t. After all, some people like to be nice during a usability testing session and are afraid to critique the website too harshly (even though you may ask them to not hold back).
To avoid those inaccuracies and get around people’s tendency toward niceness, you can also test the way people respond to speed indirectly. As an example, with in-person testing where you can control the computer or smartphone, you could rig the website to load considerably faster or considerably slower. Then, you could see how that change to the speed affects people’s ability to use the website as well as their answers to other questions and their general sense of frustration using the website during the testing session. You might find a slower website speed prevented them from successfully completing a form even though they said the speed was fine. You may also find that faster website makes people more likely to give positive feedback about the entire website, not just about the website’s speed.
We did something like this rigged website experiment a few years back to understand how people perceived a website’s speed. Among other surprises, we found that people kept describing a website’s load as fast, even after what is traditionally thought of as the cut off into the “slow” website territory. Since then, we’ve seen with various clients that many people are more forgiving of slower speeds than us web professionals would expect. So, as you start measuring the perception of speed, be ready for surprises and to change your mind about how fast your website really ought to be.
Review Competitors Speed
Another way to determine if your website is “fast enough” is by reviewing competitor websites. For the ways speed affects SEO, you want to review the speed of websites you compete with for rankings on key terms. However, more generally, you want to review the speed of websites you compete with for customers too. Your objective is to make your website faster than (or as fast as) those competitor websites. (Or, put another way: “I don’t have to be faster than the bear. I only have to be faster than you.”)
Like with the correlation to conversion or engagement metrics discussed above, a word of caution here is that this is most helpful once your website is within an acceptable baseline. Perhaps the acceptable baseline for loading is “fast enough” to have your website be faster than all other competitor websites. Or, you might determine you need to go even faster than that baseline to be able to compete—in some industries, one second load times are still too slow. But until you reach a reasonable baseline, knowing a competing website’s speed is an irrelevant data point—who cares how fast or slow a competitor’s website loads when your website is currently moving at a snail’s pace and annoying visitors?
What Features & Images Are Needed?
This last metric is less about understanding how fast a page loads but more a way to help diagnose ways we can speed up a page if we determine we must. It should come as no surprise that the more stuff you’ve got on a page, the slower it will load. So, as we think about a website’s speed the other thing we need to measure if all that stuff is being used; is that stuff worth the load time cost?
How do we answer that? Well, for starters, we need to know if people are even looking at all five images in the carousel. With event tracking scripts like Riveted, we can measure precisely how much time people spend on a page. Maybe we find on our page with that five-image carousel, people are spending 55 seconds. That seems like enough time to view all five images. However, we also need to know how much of that 55 seconds is spent on the part of the page with the carousel. Something like my UX event tracking script can help here to measure the time people scroll by certain elements. And with that, we might find that while people spend 55 seconds on the page, only 5 seconds of that time is spent where they can see the carousel. If we were to find that, it would be clear that the carousel isn’t worth the speed cost.
Of course, we can measure this in other ways too—such as a split test to compare how conversions are affected with or without the carousel. Even if people aren’t watching the carousel rotate, maybe they pick up on it indirectly and the presence of that rotation, and all the large images, is enough to encourage more conversions than would happen with a static image.
We can walk through measurements for all other kinds of features we might have on a page. Do people watch the videos contained on the page? Does that quiz or calculator get enough interaction to justify all the scripts you must load onto the page? Are people looking at and viewing the ads—and is there enough revenue generated by the ads—to make those worthwhile? With tabbed content, are people clicking on each tab to view the content or would a simpler (and faster) presentation style work better for users while also saving on load time?
When it comes to measuring speed, we need a complete understanding of load time. Tools to measure your website’s load time are important but simply knowing that the website loaded in 3 seconds doesn’t really tell us enough. Those 3 seconds might not hold the website back at all. Or that 3-second load might be snail-paced given the type of website, the competition and the type of customers involved. The only way to know for sure is by correlating the speed to engagement and conversion, understanding how people who use the website perceive the speed, and how fast the website is relative to the competition. Until you measure these factors, you won’t know for sure if your website is fast enough for visitors to your website.
Finally, the other important thing to mention when it comes to speed is Google’s Core Web Vitals which rolls out as a factor for organic search rankings starting in May 2021. Speed, and in particular total load time are important, but in addition to measuring the effects of total load time for your visitors you should also investigate the impacts of Google’s Core Web Vitals, especially Largest Contentful Paint (LCP) and First Input Delay (FID), on your rankings and user experience. Check out my comprehensive guide to Google’s Core Web Vitals to learn more.