SEO Friendly Infinite Scroll: How Do You Make Infinite Scroll Work For Organic Search?
By Matthew Edgar · Last Updated: May 20, 2022
Instead of relying on traditional pagination to present a long list of items, many websites opt for infinite or continuous scrolling. With infinite scrolling, as people scroll down the page, more items are loaded into the browser. This can create a more engaging experience for visitors. Infinite scrolling can work for many different types of websites, including websites that list products for sale, blog posts, items in a directory, and more.
However, infinite scroll can present a problem for SEO because Googlebot doesn’t scroll. If Googlebot doesn’t scroll, then Googlebot can’t see all of the listed items. So, how do you implement infinite scrolling in a way that allows Googlebot to see all the items listed on the website?
Jump to the solution for SEO-friendly infinite scroll or keep reading for more background information.
How Google Crawls: Fixed Size and Fixed State
To begin, why doesn’t Googlebot scroll down the page? Google loads a website in a headless browser at a fixed size and in a fixed state.
Fixed size. Googlebot only sees one screenful worth of content. In contrast, human visitors might make the screen larger to see more of the content, may scroll to capture more of the screen, or might zoom in to see parts of the screen more clearly. But Googlebot doesn’t do this and because Googlebot doesn’t change the size of the screen, Googlebot may miss out on certain content contained on the page if seeing that content requires zooming, scrolling, or resizing the screen. To avoid missing these details, Googlebot loads the website at a larger height, which allows their bot to capture as much of the page as possible. Last reported was that Googlebot can load a web page with a height of up to 9,307 pixels.
Fixed state. Googlebot doesn’t alter the page once loaded; the page is what it is once loaded and Googlebot won’t see it any other way. In contrast, human visitors will change the state of the page by interacting with the page. Scrolling is one type of interaction that can change the nature of the page—by scrolling you can see different content appear. Visitors can also change the state of the page by clicking on items, pressing keys, interacting with form fields, and more. I covered this more in my article about how Googlebot supports JavaScript but the main idea is that Googlebot doesn’t interact beyond the initial load of the page. This means Googlebot might miss content that only appears following some type of interaction.
Coming back to infinite scroll, we can see two main problems. As you can see in the visual below, an infinite scroll changes the state of the page. If you don’t scroll on this page, you have a very different experience with the page. Because we know Googlebot loads the website at a larger size to purposefully avoid having to scroll and because we know Googlebot doesn’t interact beyond loading the page, we know that Googlebot won’t be able to see the additional content loaded as you scroll down the page.
Load More
A related concept to infinite scroll is “load more” functionality. Similar to infinite scroll, only a finite number of items are presented on the page by default. A visitor then has to click a “load more” button which then loads additional items into the page. Instead of being triggered on scroll, the additional items are loaded in response to a click-based event. Googlebot doesn’t click on buttons and so won’t be able to interact with this functionality. Like with infinite scroll, with a “Load More” button as the only means of loading additional items to a page, Googlebot would not be able to see any of those additional items.
Solution: SEO Friendly Infinite Scroll
Whether using infinite scroll or load more, Googlebot’s inability to see additional items can present a problem for SEO. This becomes a critical problem if not seeing those items will prevent your website from ranking. For example, on a blog website’s category page, it would be important for Googlebot to see the list of all blog posts. If, however, the category page uses infinite scroll or load more functionality to load the list of blog posts, then Googlebot may not be able to see the links to all of the website’s blog posts. This can mean that Googlebot misses some of the blog posts and those blog posts are not able to rank and earn organic search traffic.
How do you make infinite scroll SEO friendly? Google discussed the solution in an older, but still relevant, article. The main idea is that you need to create a traditional paginated series to list and present items. This paginated series works alongside the infinite scroll functionality. Let’s recap the key recommendations Google has provided and review the steps to implement an SEO-friendly infinite scroll.
- Create Component Pages
- Craft Unique URLs for Paginated Pages
- Each Component Page Should Have Unique Content
- Link to Paginated Series
Note: Quotes below are from the help article from Google linked above.
Create Component Pages
“Chunk your infinite-scroll page content into component pages that can be accessed when JavaScript is disabled.”
“Test that each component page (the URL) works to take anyone directly to the content and is accessible and referenceable in a browser without the same cookie or user history.”
These two items mean that the website should have distinct and unique URLs for each page of the paginated series. That is, there should be a distinct URL for pages 2, 3, 4, and so on. As well, these URLs should load even if the visitor or robot hasn’t started on the first page of the series.
Craft Unique URLs for Paginated Pages
“Each component page contains a full URL. We recommend full URLs in this situation to minimize potential for configuration error.”
The URL for each URL page should be a URL that Googlebot is able to crawl, index, and rank. Typically, pagination is handled via parameters (for example, mysite.com/items?page=2). However, pagination can also be handled as a slug, which is how WordPress handles pagination (for example, www.matthewedgar.net/blog/page/2 /). In other cases, websites aren’t able to know a specific page number but have a range of items (for example a site that shows 25 items per page might have a URL like mysite.com/items?start=25).
What you want to avoid are URL fragments to designate a specific page. For example, you would not want to use something like mysite.com/items#2 to designate the second page of the paginated series. Googlebot wouldn’t be able to differentiate this as a unique page as this URL structure can also be used to designate areas on a page.
Each Component Page Should Have Unique Content
“Divide content so that there’s no overlap between component pages in the series”
Finally, Googlebot wants to ensure that each page in the paginated series has unique content. Page 2 should list different items than page 3. This is a way to avoid creating duplication on the website, resulting in a poor user experience.
Typically, this is not a problem for most websites. Where it can become a problem is for websites that can’t use page numbers but rely on a range instead where the ranges are flexible. As an example, mysite.com/items?start=25 and mysite.com/items?start=26 might list virtually identical lists of items with only one difference because one URL starts at item 25 and the other starts at item 26. If you are using a range instead of fixed page numbers, ensure that you lock down what ranges are accessible. In this example, mysite.com/items?start=26 should return a 404 if 26 is not a valid starting number.
As part of ensuring each page contains unique content, it is important that each page has a unique title tag and header. For example, Page 2’s title and h1 tag should likely reference the current page. For example, on mysite.com/items?page=2, the title could be something like <title>My Site’s Items | Page 2</title> and the h1 might be <h1>My Site’s Items, Page 2</h1>. Doing this helps establish that each page is unique and distinct.
Because each component page should be unique, it is typically best practice to allow each component page to be crawlable and indexable. As well, each component page should canonical to itself (the canonical on mysite.com/items?page=2 should be mysite.com/items?page=2). Note that this means component pages might rank in search results. So, if these pages will rank and bring in traffic, make sure each component page is designed appropriately to match the remainder of the website.
Link to Paginated Series
Once the paginated series and all component pages are built, the links to those component pages need to be exposed to Googlebot. How this is exposed changes for the first page of the series and the subsequent pages in the paginated series.
The first page of results is likely the only page that human visitors will see. This is the page that contains the infinite scroll functionality. It is important that along with the infinite scroll, this page contains at least a link to the second page of results, if not all pages in the series. One easy way of doing this is to show a “Next” link in a traditional <a> tag, such as:
<a href="/items?page=2">Next</a>
This link could be presented at the end of the first set of results. If a visitor scrolls down the page, triggering the infinite scroll, this “next” link can be hidden from view at the same time the next set of items are loaded onto the page. Googlebot won’t scroll or trigger the infinite scroll but will see the next link and be able to extract the link to page 2 from the link.
If you are using a “load more” functionality instead of an infinite scroll, another option for exposing the link to the second page is to combine the link to page two within the “load more” link itself. For example:
<a href="/items?page=2" id="load-more" class="button">Load More</a>
There could then be a JavaScript-based click event for this link and that click event could prevent the default behavior of loading the page to instead trigger the “load more” functionality on the page. In the example above, there would be an event listener for #load-more. If a visitor can execute click events in JavaScript, this link would trigger that #load-more event listener’s behavior but if the visitor cannot execute click events (like Googlebot) then that visitor would be taken to the second page of results instead.
The component pages of the paginated series should also contain links to the other pages in the series. This can either be presented as previous and next links or as full links to each page of the series (a link to pages 1, 2, 3, 4, 5, and so on). Importantly, though, remember to link to the main page of the series properly. Typically, the first page of the series does not include a page number. For example, if the main page is mysite.com/items, then no links to this page should be mysite.com/items?page=1.
Final Thoughts
The biggest takeaway is that Googlebot doesn’t interact with scroll- or click-based events. Googlebot crawls the web looking for links to distinct URLs and then crawls each of those distinct URLs found. Infinite scroll, by default, does not create distinct URLs and relies on scroll-based events. Load more functionality is similar but relies on click-based events. If it is important for Googlebot to see the content exposed with infinite scroll functionality, then it is important you alter the configuration of your website’s infinite scrolling to support distinct, paginated URLs that Googlebot is able to crawl.
If you need help adjusting your website’s infinite scroll or load more functionality, or need help testing it out to ensure Googlebot is crawling through the functionality correctly, please contact me.