SEO Friendly Infinite Scroll: How Do You Make Infinite Scroll Work For Organic Search?
By Matthew Edgar · Last Updated: November 08, 2023
Instead of relying on traditional pagination to present a long list of items, many websites opt for infinite or continuous scrolling. With infinite scrolling, as people scroll down the page, more items are loaded into the browser. This can create a more engaging experience for visitors. Infinite scrolling can work for many different types of websites, including websites that list products for sale, blog posts, items in a directory, and more.
Jump to the solution for SEO-friendly infinite scroll or keep reading for more background information.
How Google Crawls: Fixed Size and Fixed State
To begin, why doesn’t Googlebot scroll down the page? Google loads a website in a headless browser at a fixed size and in a fixed state.
Fixed size. Googlebot only sees one screenful worth of content. In contrast, human visitors might make the screen larger to see more of the content, may scroll to capture more of the screen, or might zoom in to see parts of the screen more clearly. But Googlebot doesn’t do this and because Googlebot doesn’t change the size of the screen, Googlebot may miss out on certain content contained on the page if seeing that content requires zooming, scrolling, or resizing the screen. To avoid missing these details, Googlebot loads the website at a larger height, which allows their bot to capture as much of the page as possible. Last reported was that Googlebot can load a web page with a height of up to 9,307 pixels.
Coming back to infinite scroll, we can see two main problems. As you can see in the visual below, an infinite scroll changes the state of the page. If you don’t scroll on this page, you have a very different experience with the page. Because we know Googlebot loads the website at a larger size to purposefully avoid having to scroll and because we know Googlebot doesn’t interact beyond loading the page, we know that Googlebot won’t be able to see the additional content loaded as you scroll down the page.
A related concept to infinite scroll is “load more” functionality. Similar to infinite scroll, only a finite number of items are presented on the page by default. A visitor then has to click a “load more” button which then loads additional items into the page. Instead of being triggered on scroll, the additional items are loaded in response to a click-based event. Googlebot doesn’t click on buttons and so won’t be able to interact with this functionality. Like with infinite scroll, with a “Load More” button as the only means of loading additional items to a page, Googlebot would not be able to see any of those additional items.
Why Infinite Scroll Is a Problem for SEO
Whether using infinite scroll or load more, Googlebot’s inability to see additional items can present a problem for SEO. This becomes a critical problem if not seeing those items will prevent your website from ranking. For example, on a blog website’s category page, it would be important for Googlebot to see the list of all blog posts. If, however, the category page uses infinite scroll or load more functionality to load the list of blog posts, then Googlebot may not be able to see the links to all of the website’s blog posts. This can mean that Googlebot misses some of the blog posts and those blog posts are not able to rank and earn organic search traffic.
Solution: SEO Friendly Infinite Scroll
How do you make infinite scroll (or load more) SEO friendly? Google discussed the solution in an older, but still relevant, article. The solution is to create a traditional paginated series that lists items. This paginated series is then enhanced by infinite scroll functionality. Googlebot can use the traditional paginated series while visitors can use the infinite scroll.
Let’s recap the key recommendations Google has provided and review the steps to implement an SEO-friendly infinite scroll.
- Create Component Pages
- Craft Unique URLs for Paginated Pages
- Each Component Page Should Have Unique Content
- Link to Paginated Series
Note: Quotes below are from the help article from Google linked above.
Create Component Pages
“Test that each component page (the URL) works to take anyone directly to the content and is accessible and referenceable in a browser without the same cookie or user history.”
These two items mean that the website should have distinct and unique URLs for each page of the paginated series. That is, there should be a distinct URL for pages 2, 3, 4, and so on. As well, these URLs should load even if the visitor or robot hasn’t started on the first page of the series.
For example, the second page of a blog listing should be available at a distinct URL, such as https://www.mysite.com/blog?page=2. When bots request this URL, then the second page of content should be listed.
Craft Unique URLs for Paginated Pages
“Each component page contains a full URL. We recommend full URLs in this situation to minimize potential for configuration error.”
The URL for each URL page should be a URL that Googlebot is able to crawl, index, and rank. Typically, pagination is handled via parameters (for example, mysite.com/items?page=2). However, pagination can also be handled as a slug, which is how WordPress handles pagination (for example, https://www.matthewedgar.net/technical-seo/crawling/page/2/).
In other cases, websites aren’t able to know a specific page number but have a range of items (for example a site that shows 25 items per page might have a URL like mysite.com/items?start=25). For this type of sequence, though, it would be important to control the start numbers. For example, start numbers can only be divisible by 25. That way you avoid a lot of redundant and unnecessary URLs from loading on the website. You would want /items?start=25 or /items?start=50 to be valid pages but you would not /items?start=32 or /items?start=56 to be valid pages.
What you want to avoid are URL fragments to designate a specific page. For example, you would not want to use something like mysite.com/items#2 to designate the second page of the paginated series. Googlebot wouldn’t be able to differentiate this as a unique page as this URL structure can also be used to designate areas on a page.
Each Component Page Should Have Unique Content
“Divide content so that there’s no overlap between component pages in the series”
Finally, Googlebot wants to ensure that each page in the paginated series has unique content. Page 2 should list different items than page 3. This is a way to avoid creating duplication on the website.
To help communicate that each page contains unique content, it is helpful to have a unique title tag and header on each page. The title and h1 tag could reference what page you are on. For example, mysite.com/items?page=2, the title could be:
<title>My Site’s Items | Page 2</title>
The h1 on page 2 might be:
<h1>My Site’s Items, Page 2</h1>
Doing this helps establish that each page is unique and distinct. Because each component page should be unique, it is typically best practice to allow each component page to be crawlable and indexable.
As well, each component page should canonical to itself (the canonical on /items?page=2 should be https://www.mysite.com/items?page=2). Note that this means component pages might rank in search results. So, if these pages will rank and bring in traffic, make sure each component page is designed appropriately to match the remainder of the website.
Link to Paginated Series
Once the paginated series and all component pages are built, the links to those component pages need to be exposed to Googlebot. How this is exposed changes for the first page of the series and the subsequent pages in the paginated series.
The first page of results is likely the only page that human visitors will see. This is the page that contains the infinite scroll functionality. It is important that along with the infinite scroll, this page contains at least a link to the second page of results, if not all pages in the series. One easy way of doing this is to show a “Next” link in a traditional <a> tag, such as:
This link could be presented at the end of the first set of results. If a visitor scrolls down the page, triggering the infinite scroll, this “next” link can be hidden from view at the same time the next set of items are loaded onto the page. Googlebot won’t scroll or trigger the infinite scroll but will see the next link and be able to extract the link to page 2 from the link.
If you are using a “load more” functionality instead of an infinite scroll, another option for exposing the link to the second page is to combine the link to page two within the “load more” link itself. For example:
<a href="/items?page=2" id="load-more" class="button">Load More</a>
The component pages of the paginated series should also contain links to the other pages in the series. This can either be presented as previous and next links (page 2 links to page 1 and page 3), as full links to each page of the series (a link to pages 1, 2, 3, 4, 5, 6, 7, and so on), or as some combination (page 2 links to page 1, 3, 4 and 5).
Remember to link to the main page of the series properly. Typically, the first page of the series does not include a page number. For example, if the main page is mysite.com/items, then no links to this page should be mysite.com/items?page=1. Attempting to load mysite.com/items?page=1 should redirect to mysite.com/items. That way, you make sure there is only one URL for the first page in the series, avoiding duplication.
The biggest takeaway is that Googlebot doesn’t interact with scroll- or click-based events. Googlebot crawls the web looking for links to distinct URLs and then crawls each of those distinct URLs found. Infinite scroll, by default, does not create distinct URLs and relies on scroll-based events. Load more functionality is similar but relies on click-based events. If it is important for Googlebot to see the content exposed with infinite scroll functionality, then it is important you alter the configuration of your website’s infinite scrolling to support distinct, paginated URLs that Googlebot is able to crawl.
If you need help adjusting your website’s infinite scroll or load more functionality, or need help testing it out to ensure Googlebot is crawling through the functionality correctly, please contact me.