📘 This article explains how to enable SpeedWorkers to serve Markdown pages to AI bots and how to preview any page in Markdown format. SpeedWorkers is part of Botify's Activation Suite, available as an option with a Botify Pro or Enterprise plan.
Overview
Since speed is crucial when AI bots perform real-time retrieval, they may deprioritize difficult-to-parse HTML pages. Serving a simplified, content-first Markdown version to AI bots maximizes the chances that AI bots can quickly access, understand, and summarize your content accurately.
You can choose to have SpeedWorkers serve Markdown to AI bots instead of HTML. While you may not want to do this for all pages, such as your product listing and detail pages, you may want to enable Markdown for your blog and content pages to provide easier to ingest content that is available sooner to bots.
👀 Enabling Markdown for AI bots instructs SpeedWorkers to serve a different representation of the page, but it is the same URL and content, so this is not cloaking.
Enabling Markdown
You can choose which AI bots receive Markdown responses from SpeedWorkers when you configure cache behaviors.
👉 Traditional bots (Googlebot, Bingbot, etc.) continue receiving fully-rendered HTML from the SpeedWorkers cache when Markdown is enabled for AI bots.
Previewing Markdown pages
Previewing pages in SpeedWorkers lets you see how pre-rendered pages appear to bots. In addition to HTML, you can view an HTML+Markdown preview or a Markdown-only preview. A Markdown preview is available for all pages cached by SpeedWorkers, not only those configured to be served to AI bots.
To see a Markdown preview of any page in the SpeedWorkers cache:
From the dropdown list, select "HTML+Markdown" or "Markdown". The page refreshes to display the selected format.
To determine if the page is configured to serve Markdown to AI bots, refer to "AI bots" in the Behavior section. When a Markdown option is enabled, hover over "Selected bots" to see which bots SpeedWorkers will serve Markdown to.
See also: