If you have enabled sitemap analysis in your project settings, the Sitemaps section will be included in the SiteCrawler reports. This report details which URLs crawled by Botify were present in or absent from your sitemaps and provides various indicators for URLs in your sitemaps. You can use this as a quick sitemap audit to find strategic, indexable pages that should be in the sitemap but are not.

The Sitemaps report includes visualizations from other SiteCrawler reports filtered by pages found in sitemaps. Since the goal of including URLs in your sitemaps is for search engines to consider indexing these strategic pages, examining their quality and performance data is important to ensure a good bot experience.

The following KPI totals for the current crawl are displayed at the top of the Sitemaps report. Click a KPI to display the corresponding URLs in a URL Explorer report:

The Sitemaps report "Top Charts" section includes the following visualizations.

This chart shows the proportion of pages found by Botify's crawler in sitemaps by their indexability status, those outside the project's scope (as defined in project settings), and those not found by Botify's crawler. You can find samples of the URLs not crawled in the Sample Of URLs Not In Crawled Structure and Sample Of URLs Out Of Project Scope charts.

Navigate to Reports > DataExports and click the Download link in the "Orphan from sitemaps" row to get a report of the URLs in sitemaps that Botify did not crawl because they were not found or out of scope.

This table displays indexability insights on pages in sitemaps based on the evolution across the compared crawls. The "# URLs" column displays the number of URLs matching the metric in the current Botify crawl, and the Change column displays the percentage of increase or decrease from the compared crawl. Click a metric to display the list of corresponding URLs in a URL Explorer report. Click the alert icon to define an alert for future changes in the corresponding metric. Click the View More link to drill into the chart in the Sitemaps Insights section.

This chart shows the distribution of pages by whether they were in sitemaps during the crawl and their location in the site by the number of clicks away from where the crawl started (typically the site's home page) using the shortest available path. The 0 on the chart's X-axis defines the crawl start page.

This is the Load Time Distribution chart from the Performance report filtered by pages in sitemaps.

This is the HTTP Status Codes Distribution chart from the HTTP Status Codes report filtered by pages in sitemaps.

This is the HTML Tags Performance For Indexable URLs chart from the Content report filtered by pages in sitemaps.

This is the Internal Follow Outlinks Destination Distribution chart from the Outlinks report filtered by pages in sitemaps. Click on the non-indexable URL segment of this chart to view a URL Explorer report of the non-indexable URLs found in sitemaps.

This is the Internal Follow Outlinks Broken Links Destination chart from the Outlinks report filtered by pages in sitemaps.

This is the Active / Not Active URLs chart from the Visits report filtered by pages in Sitemaps.

This table provides some of the URLs in sitemaps that were within the project scope, but not found by Botify's crawler. This can occur if the pages were linked from pages not crawled by Botify because they were deeper in the site structure than the crawler was set to explore or because they are orphan (i.e., unlinked) pages. To learn why these pages were not crawled, refer to the SiteCrawler Overview report. If the Crawled Pages and Known Pages are equal in the KPI banner, they are orphan pages.

If any pages were found in your sitemaps outside the crawl perimeter defined in project settings (e.g., domain/subdomain or protocol), this table includes a sample of those pages.

This table links the sitemap files found during the crawl and includes the number of URLs found in each file, regardless of whether Botify crawled them.

If errors are encountered when ingesting your sitemaps, this table lists the errors.

See also: