📘 This article provides the list of bots supported in SpeedWorkers. SpeedWorkers is part of Botify's Activation Suite, available as an option with a Botify Pro or Enterprise plan.
Overview
SpeedWorkers can receive requests from the bots identified in Supported Bots; however, you must have Bing in your contract to serve to AI bots through SpeedWorkers. Any other bot is considered unauthorized.
Bot user agents
To identify bots in logs or when configuring CDN settings, include the following user agents according to those allowed by your contract.
👉 User agent names are case-insensitive in routing rules.
Botify
botify-bot-sw-*
*adsbot-google/*
*googlebot/*
*google-inspectiontool/*
*googleother/*
Bing
*bingbot/*
Other Search bots
*yandexbot/*
*yandexmobilebot/*
*baiduspider/*
*baiduspider+/*
*applebot/*
*yeti/*
AI bots
*GPTBot/*
*Amazonbot/*
*anthropic-ai*
*Bytespider*
*CCBot/*
*ChatGPT-User/*
*Claudebot/*
*Claude-Web*
*Claude-User/*
*Claude-Searchbot/*
*FacebookBot/*
*meta-externalagent/*
*OAI-AdsBot/*
*OAI-SearchBot/*
*PerplexityBot/*
*Perplexity-User/*
*YouBot/*
👀 Wildcards are used in this list because some full user agent strings are not publicly available, and new ones are continuously added.
Identifying user agents
When identifying user agents to intercept for SpeedWorkers, use the format required by your CDN, such as:
AdsBot\/|amazonbot\/|anthropic-ai|applebot\/|bingbot\/|bytespider|ccbot\/|chatgpt-user\/|claude-web|claudebot\/|claude-user\/|claude-searchbot\/|facebookbot\/|google-inspectiontool\/|googlebot\/|googleother|gptbot\/|meta-externalagent\/|oai-adsbot\/|oai-searchbot\/|perplexity-user\/|perplexitybot\/|youbot\/|botify-bot-sw-
*adsbot* *amazonbot* *anthropic-ai* *applebot* *bingbot* *bytespider* *ccbot* *chatgpt-user* *claude-web* *claudebot* *claude-user* *claude-searchbot* *facebookbot* *google-inspectiontool* *googlebot* *googleother* *gptbot* *meta-externalagent* *oai-adsbot* *oai-searchbot* *perplexity-user* *perplexitybot* *youbot* *botify-bot-sw-*
adsbot|amazonbot|anthropic-ai|applebot|bingbot|bytespider|ccbot|chatgpt-user|claude-web|claudebot|claude-user|claude-searchbot|facebookbot|google-inspectiontool|googlebot|googleother|gptbot|meta-externalagent|oai-adsbot|oai-searchbot|perplexity-user|perplexitybot|youbot|botify-bot-sw-
Excluding agentic bots from SpeedWorkers
Agentic bots, such as Google Agent and Google Store bot, should not be served by SpeedWorkers because they are a fundamentally different traffic type from “traditional” bots.
SpeedWorkers delivers cached HTML snapshots to bots that scan pages to build an index. This replaces the need to fetch the page from the origin server, enabling SpeedWorkers to respond fast. Agentic bots mimic live user sessions, such as adding an item to a shopping cart, which requires the page’s actual current state. Anything transactional requires the origin server, which SpeedWorkers is designed to bypass.
🚀 While some of the supported AI bots are triggered by user requests inside a chat session, they respond by fetching read-only content, which SpeedWorkers can serve.
While there are too many agentic bots to list here, the most common are:
Storebot-Google: Simulates a user shopper to test the checkout flow.Google-Agent: Live user-proxy agent that takes real actions on a user's behalf.
Exclude the following bot types (when the user agent is known) as a best practice:
Full-checkout agents (e.g., Google Store bot, Google Agent).
Recommend-and-route agents: These generally operate as full browser sessions or embedded widgets without a distinguishing user agent, so CDN-level exclusion may not be possible.
Configuring SpeedWorkers for excluded bots
SpeedWorkers can only prevent serving these bots when they are excluded from your CDN configuration. Check your settings to confirm you are not routing any traffic to the agentic bots with known user agents.
Change log
Change log
June 12, 2026: Added OAI-AdsBot
June 5, 2026: Added Claude-User and Claude-SearchBot
See also: