📘 This article describes the Performance report in SiteCrawler, part of Botify's Analytics Suite, available with all Botify plans.
Overview
SiteCrawler's Performance reports display metrics related to load times experienced by bots when visiting your site. The load time metric measured by Botify in this report is the time it took for the crawler to request and download the complete HTML of each page on the website. If your report has JavaScript crawling enabled, you will see additional visualizations related to the time it took to download resources (e.g., images, font files, CSS) and render each page.
Performance Report KPIs
The following KPI totals for the current crawl are displayed at the top of the Performance report. Click a KPI to display the corresponding URLs in a URL Explorer report:
Crawled URLs: The number of pages crawled by Botify in the selected crawl.
Average Load Time (in ms): The average time it took all pages to load when crawled by Botify. Click this KPI to view the load times for individual pages in the URL Explorer.
Slow URLs (>1s): The number of pages crawled by Botify that took one second or more to load fully.
Fast URLS (<500 ms): The number of pages crawled by Botify that took 500 ms or less to load fully.
Performance Report Visualizations
The report's "Top Charts" section includes the following visualizations. Visit the Segments section of the Performance report to see many of these visualizations focused on the page categories you have defined in your project segmentation:
Load Time Distribution
This chart shows the delay between when the Botify crawler requested the URL and when the page HTML code was fully downloaded. The ideal load time is 500 ms or less. Your page load time is important for bots since a fast load time enables them to crawl more of your site, and for users, since a fast load time provides a better user experience.
Insights
This table displays performance insights based on the evolution across the compared crawls. The # URLs column displays the number of URLs matching the metric in the current Botify crawl, and the Change column displays the percentage of increase or decrease from the compared crawl. Click a metric to display the list of corresponding URLs in a URL Explorer report. Click the alert icon to define an alert for future changes in the corresponding metric.
Load Time Performance By HTTP Status Code
This chart shows the distribution of your site pages by the HTTP status code they returned during Botify's crawl. As explained in our blog article, many factors affect load time by HTTP status code in a crawl.
Load Time Performance By Depth
This chart shows the distribution of load times by page depth, or number of clicks away from the crawl start page (typically the home page).
Pay attention to lower-depth pages that are not loading fast. Higher-depth pages often load slower because of pagination or faceted navigation. If you have defined segments in your project, visit the Performance Report Segments section to evaluate your page load times by specific site sections.
Average Load Time By Language
This chart shows the distribution of load times by your page language, defined by the page HTML tag "lang" attribute.
Average Load Time By Domain / Subdomain
This chart shows the distribution of load times by the domains and subdomains defined in your project settings.
Average Load Time Gzip Vs Non Gzip
This chart shows the distribution of load times by page compression.
JavaScript Load Time
This chart shows the time it took Botify's crawler to render the page JavaScript, which occurs after the HTML load time. As with HTML load time, a fast JS load time is desirable. If errors are encountered during this rendering, they are reflected in the JS Crawl Resources Warnings chart below.
JS Crawl Resources Warnings
This table provides insight into warnings and errors encountered by Botify's crawler while rendering page JavaScript. It also reveals possible reasons for slow JS rendering, such as a large external resource size.
See also: