Skip to content

Crawling & Rendering

SEO begins with a robot crawling a website. During crawling, search engine robots open every page they have found a link to and fetch all the content from that page. As part of fetching content, robots need to fully render the pages by executing JavaScript files. You need to make sure robots are able to find all the pages and files on the website. As robots do find the pages and files, you also need to make sure robots can see the content on those pages and files. That includes making sure robots can execute important JavaScript files too. The resources below will help you find and fix crawling-related issues that are holding back your SEO performance.

How to Check HTTP Response Status Codes

Every page on every website returns an HTTP response status code. How do you check the status code for your website’s pages? What tools can you use to test status codes? What do the status codes mean?

Noindex vs. Nofollow vs. Disallow

Are you confused about the difference between noindex, nofollow and disallow commands? All three are powerful tools to use to improve a website’s organic search performance, but each has unique situations where they are appropriate to apply.

How to Use a Headless Browser

Learn what a headless browser is, why you should use a headless browser and how to launch a headless browser on your own computer.

Robot Crawlability

The most critical technical SEO task is making your website accessible to search engine robots that need to crawl through the various pages, images, and other content on your website.

Understanding Entities

How does Google understand what a name is or that a name relates to a specific person? In this post, Matthew digs into entities.

Subdirectory or Subdomain?

Should you use a subdirectory (subfolder) or subdomain for SEO and for the best user experience?

Mobile-First Indexing

Google typically crawls and evaluates the mobile version of a website first and uses what is found on the mobile website to decide where to rank the website in search results. How do you make sure your website works for mobile-first indexing?

SEO Friendly Infinite Scroll: How Do You Make Infinite Scroll Work For Organic Search?

How do you make infinite scroll and related techniques SEO friendly? See the steps you need to take to fix infinite scroll for SEO.

Website Error Pages

When you access a web page that the website cannot find, the server that powers that website usually returns a status code indicating that the server was unable to find the file requested. That status code returned when a file is not found can vary, but is most commonly a 404.

Selecting a Website’s Canonical Domain

Should you use WWW or not in your URL? What about using an SSL certificate? In this blog post, learn what a canonical, or preferred, domain is and how this can affect your SEO and user experience.

SEO Discoverability

Before you can do anything else with SEO, you need Google to find your pages. This process of finding pages is called “discoverability”. Learn about discoverability and how to make your pages easier for Google to find.

Managing URL Parameters & Query Strings: SEO Best Practices

Parameters can create duplicate content, slow down website crawling, and disrupt rankings. What are the best ways to avoid these problems? In this article, we’ll review how to find and manage parameters on a website.