📘 This article describes the Oveview report in SiteCrawler, part of Botify's Analytics Suite, available with all Botify plans.
Overview
SiteCrawler's Overview report is the default view in SiteCrawler that provides KPIs and visualizations representing the top charts from each linked report in the left navigation pane. Click the View More link below any chart for the related in-depth report section.
Overview Report KPIs
The following KPI totals for the selected period are displayed at the top of the Overview report. Click a KPI to display the corresponding URLs in a URL Explorer report:
Crawled URLs: The number of unique pages crawled by Botify in the selected crawl.
Discovered URLs: The number of unique pages found by Botify in the selected crawl. Read why this number may be higher than the number of crawled URLs.
% Active URLs: The percentage of pages crawled by Botify that received at least one user visit during the previous 30 days, as reported by your third-party analytics provider.
Visits Volume: The total number of organic visits during the previous 30 days, as reported by your third-party analytics provider.
Revenue (All Channels): The total revenue from all traffic channels during the previous 30 days, as reported by your third-party analytics provider.
Overview Report Visualizations
The following visualizations are included in the Overview report. Except for the "Active/Not Active URLs" chart, these visualizations summarize more detailed reports in SiteCrawler.
Active/Not Active URLs
This chart displays the distribution of all pages by organic traffic performance with the indexable status of inactive pages, derived from Visits and Distribution reports. This chart clearly shows which pages are successful because they generated organic traffic and which pages need attention. You should evaluate the inactive pages shown here to determine if there are opportunities to optimize these pages to avoid wasting your crawl budget or a bad user experience.
Click the View More link to drill into this chart in the Visits reports.
Active: Pages that received at least one user visit during the previous 30 days.
Indexable Not Active: Pages eligible to be crawled by search engines that did not receive at least one user visit during the previous 30 days.
Non-Indexable Not Active: Pages ineligible to be crawled by search engines that did not receive at least one user visit during the previous 30 days. A page is non-indexable if it does not serve a 200 HTTP status code, has a noindex meta tag, canonical tag pointing to another URL, or a content type other than text/HTML.
Insights
This table displays indexability insights based on the evolution across the compared crawls. The # URLs column displays the number of URLs matching the metric in the current Botify crawl, and the Change column displays the percentage of increase or decrease from the compared crawl. Click a metric to display the list of corresponding URLs in a URL Explorer report. Click the alert icon to define an alert for future changes in the corresponding metric.
Indexable URLs: Pages eligible to be served to search engines.
Non-Indexable URLs: Pages ineligible to be served to search engines.
2xx URLs: Pages that returned 2xx HTTP status codes during Botify's crawl.
Indexable URLs with Bad H1: Pages with a missing H1 tag or have an H1 tag that is a duplicate of an H1 tag on another indexable page in the same domain.
Indexable URLs with Bad Description: Pages with a missing description or have a description that is a duplicate of a description on another indexable page in the same domain.
URLs with 1 Follow Inlink: Pages containing one unique follow inlink.
Non-Indexable URLs Main Reason
This chart shows the primary reason the URLs are non-indexable, according to the following priorities: bad HTTP status code, meta noindex, canonical not equal, bad content type. Click the View More link to drill into this chart in the Distribution reports.
Meta noindex: Pages with a noindex meta tag.
Bad HTTP Code 301: Pages that returned a 301 HTTP status code.
Bad HTTP Code 500: Pages that returned a 500 HTTP status code.
Canonical Not Equal: Pages that contained a canonical tag pointing to a different URL.
Bad HTTP Code - 102: Pages that returned a 102 HTTP status code.
HTML Tags Performance for Indexable URLs
This chart shows the distribution of HTML tag characteristics of all indexable URLs in the same zone (i.e., the combination of domain and language from the page's "lang" tag). Click the View More link to drill into this chart in the Content report.
Unique: Indexable pages with unique tags.
Duplicate: Indexable pages with duplicate tags.
Not Set: Indexable pages with missing tags.
HTTP Status Codes Distribution
This chart shows the HTTP status codes returned to Botify's crawler, visualizing the number of pages that are delivered successfully, redirected, or return errors. Click the View More link to drill into this chart in the HTTP Code reports.
URLs By Depth And Content Type
This chart shows the distribution of your page content type (e.g., HTML, PDF, image) by the number of clicks away from where the crawl started (typically the site's home page) using the shortest available path. The 0 on the chart's X axis defines the crawl start page. Click the View More link to drill into this chart in the Distribution reports.
Load Time Distribution
This chart displays the delay between when the Botify crawler requested the URL and when the page HTML code was fully downloaded. Click the View More link to drill into this chart in the Performance reports.
Average Number of Follow Inlinks by Percentile of URLs
This graph shows whether internal links are distributed to a wide range of URLs or a small subset, using the average number of internal links found in the current crawl. Internal links with nofollow tags are excluded from this graph. Click the View More link to drill into this graph in the Inlinks reports.
Structured Data Distribution
When your pages include structured data, these charts show the structured data types' distribution in the current crawl.
URLs Distribution in Sitemaps
When you have the Sitemap comparison option enabled, this chart shows the URLs found by Botify's crawler by indexable status and the URLs in your sitemaps that were not found in the crawl. Click the View More link to drill into this chart in the Sitemaps reports.
Indexable URLs in Sitemaps: Indexable URLs identified in your sitemaps and crawled by Botify.
Non-indexable URLs in Sitemaps: Non-indexable URLs identified in your sitemaps and crawled by Botify.
URLs not in Structure: All URLs identified in your sitemaps that Botify did not find.
URLs out of Project Scope: URLs identified in your sitemaps but not crawled by Botify because they were not in the scope defined in your crawl settings.
See also: