How quickly your web pages load is crucial to increasing organic search traffic, especially for mobile. After Google’s July 2018 “Speed Update” algorithm change, improving page load time may significantly impact organic traffic for some websites. As Google prioritizes mobile-first indexing, page speed is increasingly important for enterprise websites with many mobile visitors.

Page load times are one of the main factors in how fast a search engine can crawl a website, especially when the website is a large enterprise site. When sites load fast, they are more accessible to search engines and users. Fast load times support a positive user experience and can lead to more conversions, higher pages per session, and more content consumed when users are on your site. Focus on page load times as one of the main methods of improving crawl rates and efficiency from search engines.

Learn how to create a report of slow-loading pages.

Being crawled is the first step to being indexed and visited through Google. On average, enterprise websites have a crawl ratio of only about 51% (the percentage of pages that search engines crawl). That means half of your website may be unknown to Google simply because the search engine has insufficient time to explore it. When pages are not crawled and, therefore, not indexed, the opportunity for organic traffic is lost at the second step in the organic traffic funnel.

Improving page load times is an opportunity to increase Google’s crawl volume and see more of your indexable pages crawled and indexed. In the graphs from Google Search Console (crawl stats) below, a website saw its crawl volume increase immediately as performance improved (i.e., the time spent downloading a page):

When Google can crawl pages faster, it can index more pages within the same time. Google indicates that 40% of consumers bounce from pages that take more than three seconds to load. While improving your page speed will increase your crawl ratio over time, you will likely not see results overnight. The concept of crawl budget is largely based on several criteria that take time to build up, including overall popularity (PageRank and external linking, authority, history, etc.) and content quality/amount.

Performance metrics in Botify correspond to the load time for the page's HTML code or time elapsed between the moment the crawler requested the page and when it received the full page content. This differs from page load time from a user's perspective, which is the load time for the page and all its associated resources (including images, CSS files, etc.). This means that Botify's performance metrics can be compared with the average load time information found in Google Webmaster Tools, which shows a daily average for all pages crawled by Googlebot.

Many factors can impact web performance, including whether your site uses a CDN, your users' geographic location, or the downloaded content's size. If your site is experiencing slow HTML load times, consider checking that your site is hosted in a way that makes it easily and quickly accessible for users from all locations within your target markets, and ensure that the HTML documents on your website are not excessively large.

Visit SiteCrawler's Performance report to view details of your site's performance:

Other key page-load metrics in Botify that measure page rendering performance include:

These are some examples of how the full page load time is broken down and understood in more granular segments by search engines. It is important to know how your website handles these load-time metrics since it is possible that one or two variables could be holding back the overall page performance. Use Google Lighthouse or Google PageSpeed Insights to analyze priority page templates for your website to identify how your website metrics are performing.
​

Above are the results of a Pagespeed Insights speed test. This provides you with a sense of what is good (green), what needs improvement (red), and what is in between (orange). It is a great place to start your page-load analysis.

Once you analyze your website’s page templates and document your current speed performance, we recommend using Google Lighthouse or Pagespeed Insights for their recommended performance benchmarks for each metric. It is important to augment using industry benchmark page load times to evaluate your website with other indicators (e.g., HTTP status code, page depth, subdomain) because the average may include outliers. For an example of page load time crossed with another indicator, navigate to the "Load Time Performance by HTTP Status Code" chart in SiteCrawler's Performance report to see your site's performance distribution:

See how to find out which types of pages are slow in a step-by-step guide.

Users and robots do not have the same response time on the same page. The graphs from Google Webmaster Tools and the page download performance shown in your speed analysis tools show performance experienced from the robots’ perspective — the download time for the HTML page without associated resources, such as images. Again, this is closely correlated to how Google crawls your site. This does not mean that performance from a user's perspective does not matter.

Users dislike slow pages and will bounce in high numbers, especially on mobile devices. While Google analyzes page load time, it also takes the overall user experience into account. Over time, this has become one of the top-ranking factors for Google. Google's Page Speed tools can help check performance from a user's perspective.

See also: