A robots.txt file is overriding your SEO plugin

What we observed

There is a real robots.txt file in your site’s root folder, and separately, one or more plugins are trying to generate a robots.txt through WordPress. Only one of them can win, and the file always does.

Why it matters

WordPress doesn’t come with a robots.txt file. When a request comes in for one, WordPress builds it on the spot, and plugins add their rules at that moment. That only happens when no real file exists. If a file is present, your web server hands it over directly and WordPress is never involved.

So the settings screen in your SEO plugin still works, still saves, still shows your rules — and has no effect whatsoever. This is the most quietly misleading configuration in WordPress: everything looks correct and nothing is.

How to fix it

You have two choices, and which is right depends on whether you want the file.

Option A — let WordPress take over. Delete the robots.txt file from your site’s root folder. Your SEO plugin’s rules take effect immediately, with no further action. Copy anything worth keeping out of the file first, then add those rules through the plugin instead.

Option B — keep the file and edit it directly. If something outside WordPress depends on that file, keep it and treat it as the only place robots rules live. Stop using the plugin’s robots settings entirely, because they will mislead whoever looks at them next.

Do not do both. Splitting your rules between a file and a plugin is how sites end up blocking something nobody remembers deciding to block.

How to check it worked

Visit https://example.com/robots.txt in a browser. If you chose Option A, the content should now match what your SEO plugin shows. If it does not, the file is still there — check for robots.txt in the same folder as wp-config.php, and remember that a cache (a caching plugin, your host or a CDN) may keep showing the old copy for a few minutes.