Common Issues

Site Optimizer Not Crawling Your Website

If you’re experiencing issues with the Site Optimizer not analyzing your website or crawling any of your pages, the most common reason is that your sitemap is either missing or inaccessible.

The Site Optimizer relies on your sitemap to understand the structure of your website and identify which pages need to be analyzed. Without a working sitemap, the tool can’t proceed.

Here’s what to do:

1. Check your Sitemap:
   • Go to yourwebsite.com/sitemap.xml
   • If you see a list of links, you’re good. If you get an error (like 404), the sitemap doesn’t exist or is blocked.

2. Fix or Create Your Sitemap
   • WordPress: Install an SEO plugin (Yoast or Rank Math), enable its sitemap feature, and confirm yourwebsite.com/sitemap.xml loads.
   • Other Platforms: Generate a sitemap (via a plugin or online generator) and place it in your site’s root folder.

3. Double-Check Firewalls/Plugins
   • Make sure nothing on your site (e.g., security plugin) blocks access to sitemap.xml.

Once your sitemap is live and accessible, the Site Optimizer should crawl your pages normally.