robots.txtYour site may be blocking AI crawlers outright
GPTBot, ClaudeBot, PerplexityBot and Google-Extended each need explicit permission. Most small business sites were built before these existed, so the file says nothing about them — and several platforms block them by default.