Your .htaccess blocks AI crawlers, but your server ignores that file

What we observed

Your .htaccess file contains rules naming AI crawlers — BrowserMatchNoCase, SetEnvIf User-Agent or a RewriteCond on the user agent. Your web server is not Apache, and only Apache reads .htaccess.

The rules are text in a file nothing opens.

Why it matters

This cuts both ways, and both are worth knowing.

If you wanted those crawlers blocked, they aren’t. They have been getting your pages in full the whole time.

If you did not want them blocked — the rules came with a theme, a tutorial or a previous developer — then nothing is wrong right now. But the moment your site moves to an Apache host, those rules switch on and you disappear from AI answers with no obvious cause. That’s a genuinely nasty failure, because nobody connects moving to a new host with AI crawlers suddenly being blocked.

WordPress writes a .htaccess file whether or not your server reads one, which is why the file exists on nginx sites at all and why this catches people out.

How to fix it

If you want the crawlers blocked, the rule has to move to where your server actually reads its settings. On nginx that is the “server block” in its configuration, which on most managed hosts means asking your host or using their control panel. A robots.txt rule is the simpler route for well-behaved crawlers, though it only asks them to stay away; it can’t force them.

If you do not want them blocked, delete those lines from .htaccess now, while they are harmless. Leaving an inactive rule in place is leaving a trap for your future self.

Either way, do not edit the block between # BEGIN WordPress and # END WordPress — WordPress regenerates that section and your changes will vanish.

How to check it worked

Re-run the check. If you removed the rules, the finding disappears. If you moved them to your server configuration, the crawler self-test should now show those crawlers being treated differently from a browser — which means the block is really working.