This run doesn't count — your own site slowed down our test for sending too many requests too quickly
The normal browser request we send for comparison got different results before and after the crawler requests, so no comparison from this run can be trusted.
What we observed
Every self-test run sends a plain browser request before the crawler requests and an identical one after. If the site is behaving consistently, those two must match. On this run they did not.
That means something changed mid-run — almost always because our own burst of requests set off a rate limiter (a setting that slows or blocks visitors who send many requests quickly). So we threw the run away. Any difference we measured between a browser and a crawler could just as easily have been the difference between request one and request ten.
Why it matters
This is how other tools end up confidently reporting blocks that aren’t there: send ten requests quickly, get throttled halfway through, and report the crawlers that happened to come last as “blocked”. You would then go hunting for a block on GPTBot that does not exist.
We would rather report nothing than report that.
How to fix it
Usually nothing is wrong with your site — a rate limiter doing its job is a rate limiter working correctly. Try in this order:
-
Simply run it again, ideally a few minutes later and when the site is not busy. Short-term throttling clears on its own.
-
Check whether your cache, not a rate limiter, caused the difference. The first comparison request may have found the page not yet cached and the second found it cached, and that alone can change headers and timing. Running again usually settles it, because both comparison requests then get the cached page.
-
If it fails repeatedly, look at what is doing the limiting and temporarily relax it for your own server’s IP address while you test:
- A security plugin’s rate limiting or “throttle aggressive crawlers” setting.
- Cloudflare Rate Limiting rules, or Bot Fight Mode.
- Host-level limits (LiteSpeed, nginx
limit_req) — you’ll need to ask your host about these.
Put the setting back afterwards. It is protecting you.
How to check it worked
Re-run the check. When a run finishes and both comparison browser requests match, this finding is replaced by a result for each crawler. If you keep seeing this specific finding, the limiter is strict enough to deserve a look on its own — a crawler fetching several of your pages in a row may be getting the same treatment our test did.