📘 This article describes how SpeedWorkers integrates with your Content Delivery Network (CDN). SpeedWorkers is part of Botify's Activation Suite, available as an option with a Botify Pro or Enterprise plan.
Overview
SpeedWorkers is a service designed to deliver web pages to search engine crawler bots as fast as possible. It enables search engines to index more pages within the same amount of time spent on your website than without SpeedWorkers. SpeedWorkers must be deployed at the CDN level.
SpeedWorkers pre-renders your JavaScript pages in advance, at scale, and delivers them to search engine bots in a few hundred milliseconds. Pre-rendering pages enables search engines to index your pages faster and ensures the bots see the full page content. By keeping all pages in cache, SpeedWorkers delivers long-tail pages as fast as any other page.
Supported CDNs
SpeedWorkers can be implemented with any CDN that has the following capabilities:
Redirect all bot traffic to the SpeedWorkers origin based on the user-agent header
Handle all asset traffic (e.g., CSS, JS, images)
Disable content caching when the SpeedWorkers origin responds
Support authentication
Refer to the SpeedWorkers Implementation documentation for more information. Please contact Support for assistance with configuring your CDN for SpeedWorkers.
How SpeedWorkers Works with Botify Analytics
Since Botify's SiteCrawler analyzes your website’s structure and content based on what bots can see, you should configure Botify's SiteCrawler to hit the SpeedWorkers cache first. This allows most pages to be delivered to SiteCrawler by SpeedWorkers, which can reduce the load on your infrastructure. To configure SiteCrawler for SpeedWorkers, please contact Support.
Read next: