Some of your content never reaches an AI crawler

What we observed

We compared the words in your editor with the words present in the HTML your server sends, and found a gap. Your page looks complete in a browser. A crawler that does not run JavaScript sees considerably less.

Why it matters

AI crawlers fetch pages, but they don’t run JavaScript. Anything your page builds in the browser — a loop widget pulling in posts, a tabbed section, content revealed on scroll, an app built in JavaScript — is invisible to them, however good it looks to a visitor.

This is not the same as Google. Googlebot does run JavaScript, though it can take a while. So a page can rank perfectly well and still be unreadable to every AI assistant, which is exactly the kind of problem that goes unnoticed for a year.

How to fix it

Find out what is missing first. View the page source — Cmd+U in most browsers, or curl -s https://example.com/your-page | less. That is what a crawler receives. Search it for a sentence you can see on the page in your browser. If it isn’t there, that content is built in the browser, and crawlers don’t see it.

Then, depending on the cause:

Page builder dynamic widgets — Elementor loop grids, post widgets, Divi and WPBakery equivalents. These are often built on the server by default, which crawlers can read, but not always. Where a widget offers a “skin” or a rendering option, the static (non-dynamic) one is usually readable. Where it doesn’t, the lasting fix is to write the important content directly into the page instead of having the widget pull it in automatically.

Lazy rendering from a performance plugin. WP Rocket’s lazy render and similar features hold content back until the visitor scrolls to it. Exclude your main content area, or turn the feature off for pages where the text is the point.

A client-side app. If your theme or a section is built with React or Vue and only runs in the browser, fixing it properly means changing how the site is built. In the short term, make sure the page’s main point — what this page is about, in plain sentences — is written into the HTML, not added by JavaScript.

The thing to keep in mind: you do not need the whole page to be static. You need the part that answers a question to be in the HTML.

How to check it worked

curl -s https://example.com/your-page | wc -w

Compare that number with the rough word count in your editor. Or re-run the check: the Pages screen shows the before and after numbers per page, and the post editor sidebar shows the same figure while you are working on the page.