Are AI bots finding the content on your website? They don’t find links the same way Google does. So, adding a link to a page isn’t always enough. Here is what else to check…
1) Rendering and code structure
Let’s start with the basics. Bots have to be able to crawl the page where the link appears. Bots also need to be able to find the links in that page’s code.
If the discovery source (the page with the link) uses JavaScript that bots cannot execute, the links will not be visible in the final rendered HTML. That means the linked pages will not be discovered.
JavaScript has long created problems for SEO, but it creates several problems specifically for AI bots since those bots do not currently load JavaScript. So, if links to a page are loaded with JavaScript, that link will not be found by AI bots.
2) Buttons vs Links
Not all clickable elements are real links. There are two main types of clickable elements: <button> and an <a> tag.
Bots rely on <a> tags with href attributes to discover pages. A <button> does not have an href, so it cannot be a discovery source. If you are using <button> tags that rely on JavaScript to trigger navigation, bots can’t follow those.
3) Structural reasons
Google’s Quality Rater Guidelines differentiate between main and supporting content. Main content is the core content on a page. Supporting content is everything else, including the navigation, footer, calls to action, sidebars, etc.
Google’s bots give less weight to links in supporting content. If the only point of discovery is links within supporting content, the page may not be discovered. It isn’t clear if OpenAI, Perplexity, Meta-ExternalAgent and other AI bots do this, but there is some indication they might.
4) Link relevance
Search and AI bots rely on content. How relevant is this link to the page’s topic? A link surrounded by unrelated text or with vague or generic anchor text might be ignored or given less value.
In some cases, it truly is an irrelevant link that was added to the page (by mistake or to manipulate bots). In other cases, the link is relevant but the anchor text makes it appear that it wasn’t. Use relevant terms or entities in the link text to help bots understand the connection.
5) Link quantity and diversity
One link is not enough. The more (relevant) links pointing to a page, the more likely bots are to find and prioritize it. I think we all get that.
However, it is also important to avoid silos. If you only link to a new blog post from other blog posts, you are limiting discovery and limiting the context of the page being linked to. Adding links to a page from different sections of the website (homepage, product or service pages, about section, and so on) show bots that a page being linked to is useful in multiple contexts.
Recap
- Avoid JavaScript-loaded links that bots can’t render
- Use <a> tags with href attributes vs. buttons
- Prioritize links in the main content area
- Write relevant anchor text
- Add multiple links from diverse areas of your site