📘 This document explains the configuration requirements for the force-fail variant (Only GET/HEAD/OPTION) implementation of SpeedWorkers with CloudFront and VPC origins.

Overview

SpeedWorkers delivers prerendered pages to search engine and AI bots. This guide describes how to deploy it on an Amazon CloudFront distribution using Lambda@Edge to route bot traffic, with CloudFront’s native origin failover as the fallback mechanism.

In this variant, the origin-request lambda carries no configuration of your origin: when a request must not reach SpeedWorkers, the lambda answers a 403 that matches the origin group’s failover criteria, and CloudFront retries the group’s secondary origin — your true origin, defined in the distribution. The exact same lambda code therefore works for any origin, any distribution. See the redirect variant for the alternative where the lambda rewrites the origin itself, and About SpeedWorkers on CloudFront with Lambda@Edge for how to choose between the two.

When to choose this deployment

CloudFront accepts one single function per event type on a cache behavior, and viewer events cannot mix CloudFront Functions with Lambda@Edge. Choose this Lambda@Edge deployment when:

Your distribution already runs Lambda@Edge functions (authentication, redirects, A/B testing, header manipulation…) on the behaviors that serve your HTML pages.
Your bot detection needs capabilities beyond CloudFront Functions: outbound HTTP calls (for example to an external bot verification service), a full Node.js runtime, or larger code.

And choose the force-fail variant over the redirect one when you want a single, origin-agnostic lambda (several origins or distributions, origin settings managed in CloudFront only) and accept a second lambda invocation on every non-bot cache miss.

Note that CloudFront only offers GET/HEAD (or GET/HEAD/OPTIONS) as allowed methods on a cache behavior that targets an origin group — the all-methods option is not available. Routes that receive other methods (POST, PUT, …) must therefore be covered by dedicated behaviors placed before the SpeedWorkers one, targeting your origin directly. This matters twice in this variant: CloudFront only fails over for GET/HEAD/OPTIONS, so a forced 403 on another method would reach the viewer instead of your origin.

How it works

The cache behavior targets an origin group: primary = SpeedWorkers, secondary = your true origin. CloudFront’s native origin failover handles every fallback — both when SpeedWorkers cannot serve a page (cache miss, error, timeout) and when the origin-request lambda decides the request must not reach SpeedWorkers at all.

Two Lambda@Edge functions drive the routing:

Function	Event	Role
viewer.mjs	Viewer request	Classifies the request bot/user (x-sw-request-type header, used to partition the cache) and saves the original host and user-agent — this is also where you can plug your own bot verification logic
`origin_force_fail.mjs`	Origin request	Routes the request: eligible bot GET/HEAD → origin group untouched (SpeedWorkers + failover); everything else → generated 403 that forces the failover to your origin

The forced-failover round trip: on the first invocation (primary origin = SpeedWorkers), the lambda answers a 403 for ineligible requests; CloudFront matches it against the failover criteria and sends the same request to the secondary origin; the lambda is invoked a second time (origin = your true origin) and passes the request through untouched. Viewers never see the 403 — unless the secondary origin itself is unreachable, in which case its explanatory body helps diagnosis.

Routing decision table:

Request	Path taken
Bot, GET/HEAD, page URL	SpeedWorkers, failover to your origin on error/timeout/cache miss
Bot, GET/HEAD, ignored path (static assets…)	Your origin, via a forced failover
User (or unconfirmed bot)	Your origin, via a forced failover
Any client, POST/PUT/PATCH/DELETE/OPTIONS	Your origin, through the dedicated behaviors placed before this one
WebSocket upgrade	Your origin, via a forced failover

A 403 from SpeedWorkers is normal — it is the cache-miss signal that triggers the failover to your origin, exactly like the 403 the lambda generates for ineligible requests. See “Disable error response caching” below: if CloudFront caches those 403s, normal pages break.

Prerequisites

Values provided by Botify:

The SpeedWorkers origin domain (XXX.speedworkers.com)
Your website ID (x-sw-website-id)
Your token (x-sw-token)

AWS requirements:

Lambda@Edge functions must be created in us-east-1 and associated as a published version (not $LATEST).
Both functions need an IAM role trusted by lambda.amazonaws.com and edgelambda.amazonaws.com:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": ["lambda.amazonaws.com", "edgelambda.amazonaws.com"]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Permissions to create/modify CloudFront origins, origin groups, behaviors, cache policies, origin request policies and error pages.
A currently supported Node.js runtime (Node.js 24.x at the time of writing — 18.x and 20.x are deprecated on Lambda and can no longer be used for new functions).

Cost note: The viewer-request lambda runs on all incoming requests of the behaviors it is attached to. The origin-request lambda runs on every cache miss — and twice for every non-bot cache miss (once to force the failover, once on the secondary-origin pass). If most of your HTML traffic is human and misses the CloudFront cache, this roughly doubles the origin-request invocations compared to the redirect variant. Only associate the lambdas with behaviors that serve HTML pages.

Step 1 — Create the SpeedWorkers origin

In your distribution, create a new origin:

Origin domain: the SpeedWorkers endpoint provided by Botify
Protocol: HTTPS only
Name: SpeedWorkers
Connection attempts: 1
Connection timeout: 2 seconds
Response timeout: 2 seconds
Keep-alive timeout: 30 seconds
Custom headers (required — use the value nolambda until the lambdas are deployed, so misrouted requests are rejected by SpeedWorkers instead of silently served):
- x-sw-website-id: your website ID
- x-sw-token: your token
- x-sw-uri: nolambda

Short timeouts matter: they bound the extra delay a bot experiences when SpeedWorkers is unavailable before CloudFront fails over to your origin.

Step 2 — Create the origin group

Create an origin group:

Primary origin: SpeedWorkers
Secondary origin: your true origin
Failover criteria: 500, 502, 503, 504 and 403 (do not include 404)
Name: OriginGroup-SW-<YourOriginName>

403 is doubly load-bearing in this variant: it is the SpeedWorkers cache-miss signal, and the status the origin-request lambda generates to route ineligible requests to your origin. The secondary origin of this group is the only place your origin is configured — the lambda has no knowledge of it.

Step 3 — Deploy the viewer-request lambda

Create a Lambda function in us-east-1 from viewer.mjs:

Runtime: the most recent Node.js runtime available (Node.js 24.x at the time of writing)
Memory: 128 MB
Timeout: 1 second (the code only manipulates headers — raise it to 2-3 seconds only if you add your own verification HTTP call)
Include body: not checked

The template classifies requests with the user-agent pattern (swBotPattern) only. If you need to verify bot requests with your own logic — for example an HTTP call to your bot management provider — integrate it yourself in viewer.mjs: a comment in the handler marks the insertion point, right after the user-agent match. When your verification fails, return the request as-is (it stays classified as a user and reaches your origin). Two things to decide consciously if you add such a call:

Timeout: the call is on the critical path of every bot request. Keep it aggressive (500–1000 ms) and well under the lambda timeout.

Failure policy: on provider outage (network error, timeout, 5xx), either trust the user-agent match (fail-open — recommended, protects SEO) or treat the request as a user. A 4xx from a provider that is up is a decision, not an outage — don’t fail open on it.

The lambda strips every inbound x-sw-* header before doing anything else, so clients cannot spoof the classification (x-sw-request-type) or the URL rebuild (x-sw-host). The only exceptions are X-SW-Options / X-SW-Options-Auth: they drive the validation tests below and SpeedWorkers authenticates them itself.

The file ships with the test pattern (botify-bot-sw-) active: only Botify’s synthetic test bot is intercepted, so real bot traffic keeps hitting your website while the integration is being validated. Once validation passes, switch to the production pattern — it is provided in the file, commented out above the test line. If you added your own verification, make sure it recognizes the test user-agent as a bot, otherwise the validation tests will be routed to your origin.

Publish a version of the function, then associate it with your behavior on the Viewer request event.

Step 4 — Deploy the origin-request lambda

Create a second Lambda function in us-east-1 from origin_force_fail.mjs:

Runtime: the most recent Node.js runtime available (Node.js 24.x at the time of writing)
Memory: 128 MB
Timeout: 1 second
Include body: not checked

Configure the constants at the top of the file:

swPrimaryOriginDomain: the SpeedWorkers domain — must match the domain of the origin group’s primary origin exactly (this is how the lambda detects the second invocation that follows a failover, native or forced)
swAllowedUrls: optional allow-list of URL prefixes
swRewriteOrigin: optional host rewrite for staging tests

That’s all — your origin is never configured in the lambda. If your true origin changes (domain, port, timeouts), you only update the origin group’s secondary origin in CloudFront.

This lambda fails closed: it only lets a request reach SpeedWorkers when the x-sw-request-type header set by the viewer lambda is present and marks a bot. If the origin request policy does not forward X-SW-Request-Type, every request goes to your origin through the forced failover — a visible symptom (“bots not served by SpeedWorkers”) rather than a silent bypass of the bot classification. Note that this protects against misconfiguration, not attackers: the viewer lambda is what strips a client-forged x-sw-request-type, so always verify the viewer association is in place. The lambda also wraps its routing in a try/catch that forces the failover on error, so a bug degrades to “no SpeedWorkers” instead of broken pages.

Publish a version, then associate it with the same behavior on the Origin request event.

Step 5 — Configure the cache behavior

On the behavior serving your HTML pages:

Origin or origin group: the origin group created in Step 2
Allowed HTTP methods: GET, HEAD — CloudFront does not offer the all-methods option on a behavior that targets an origin group, and the forced failover only works for GET/HEAD(/OPTIONS) anyway. Routes that receive POST/PUT/PATCH/DELETE must be covered by dedicated behaviors placed before this one, targeting your origin directly; a non-GET/HEAD request whose path is only matched by this behavior is rejected by CloudFront with a 403
Cache policy (if you use the CloudFront cache): create a policy that includes the X-SW-Request-Type header in the cache key. This partitions the cache between bots and users — without it, human visitors can be served bot-prerendered pages. Do not use the legacy cache settings with “forward all headers” (which disables caching): forwarding all headers through the legacy settings also breaks the forced failover — use a cache policy and an origin request policy instead.
Origin request policy: forward at least
- X-SW-Request-Type
- X-SW-Host
- X-SW-User-Agent
- X-SW-If-Modified-Since
- X-SW-Options, X-SW-Options-Auth (used by the validation tests)
- all query strings your pages depend on (at minimum the ones that change page content)
  
  The X-SW-* headers are added by the viewer-request lambda — except X-SW-Options/X-SW-Options-Auth, which are sent by the client and merely preserved by it; if the origin request policy does not forward them, the origin-request lambda never sees them and every request falls back to your origin. Make sure your true origin accepts the requests CloudFront sends it on the failover pass — a mismatched forwarded Host header is the classic cause of an unreachable secondary origin.

Function associations: viewer request → viewer lambda version (Step 3), origin request → force-fail lambda version (Step 4).

Origin-group failover only triggers for GET/HEAD/OPTIONS viewer requests, which matches this behavior’s allowed methods. Inventory the routes under this behavior’s path pattern that receive other methods (forms, APIs, carts…) and create their dedicated behaviors before enabling the SpeedWorkers one — behavior selection is by path only, first match wins, and a method not allowed by the matched behavior is rejected rather than falling through to the next one.

Step 6 — Disable error response caching

CloudFront caches error responses for 10 seconds by default. This variant relies on 403 responses twice — the SpeedWorkers cache-miss signal and the lambda’s forced failover; if a 403 gets cached, CloudFront serves it again instead of retrying, and pages break.

In the distribution’s Error pages tab, create a custom error response for each of 403 (critical), 400, 404, 500, 502, 503, 504 with:

Error caching minimum TTL: 0
No response page override

Integrating with existing Lambda@Edge functions

If the behavior already has a lambda on the viewer-request or origin-request event, merge the SpeedWorkers logic into it (CloudFront accepts a single function per event):

Viewer request: run the SpeedWorkers classification (the viewer.mjs handler body) first, before any of your logic rewrites the user-agent, host or URI. Your existing logic then applies to the returned request as usual.
Origin request: run your existing origin-selection logic first, then the SpeedWorkers routing (route() in origin_force_fail.mjs) last. Careful with the second invocation: on the failover pass your existing logic runs again too — make sure it is idempotent.
Keep the inbound x-sw-* header stripping at the very top of the merged viewer function.

Testing and validation

Run the three standard SpeedWorkers tests with the test user-agent botify-bot-sw-test (the viewer lambda ships with the matching test pattern active):

Test 1 — Cache hit (always success)

curl -i -A "botify-bot-sw-test" -H "X-Sw-Options: passed-through,request-time,always-success,echo-67674" -H "X-Sw-Options-Auth: <WEBSITEID>" https://www.yourdomain.com/

Expected: status 200, body Success, and the response headers X-Sw-Status: success, X-Sw-Echo: 67674, X-Sw-Passed-Through: true.

Test 2 — Cache miss (always notfound)

curl -A "botify-bot-sw-test" -H "X-Sw-Options: passed-through,request-time,always-notfound" -H "X-Sw-Options-Auth: <WEBSITEID>" https://www.yourdomain.com/

Expected: status 200, your origin’s homepage, no X-Sw-* response headers (SpeedWorkers answered 403, CloudFront failed over to your origin).

Test 3 — Timeout (always timeout)

curl -A "botify-bot-sw-test" -H "X-Sw-Options: passed-through,request-time,always-timeout" -H "X-Sw-Options-Auth: <WEBSITEID>" https://www.yourdomain.com/

Expected: status 200 after a short delay (the SpeedWorkers response timeout), your origin’s homepage, no X-Sw-* headers.

Three additional tests specific to this deployment:

Test 4 — Forced failover for users

curl -i https://www.yourdomain.com/

Expected: status 200 and your origin’s homepage, with a regular user-agent. If instead you receive a 403 whose body starts with Forcing fallback to secondary origin!, the forced failover fired but CloudFront could not get a valid response from the secondary origin — check the origin group’s secondary origin configuration and the forwarded Host header.

Test 5 — Non-GET methods bypass SpeedWorkers

curl -A "botify-bot-sw-test" -X POST -d "a=1" https://www.yourdomain.com/your-form-endpoint

Expected: your origin’s normal response, served through the endpoint’s dedicated behavior. A POST to a path matched only by the SpeedWorkers behavior returns a 403 from CloudFront (the behavior only allows GET/HEAD) — if that happens on a legitimate route, it is missing its dedicated behavior.

Test 6 — Cache partitioning

Request the same URL twice, once with a regular browser user-agent and once with the test bot user-agent. Each must receive its own version; check that the browser response never carries SpeedWorkers markers.

Botify runs an automated batch validation after deployment — contact your CSM when the staging behavior is ready.

Troubleshooting

Symptom	Likely cause	Resolution
403 with body “Forcing fallback to secondary origin!”	The secondary origin is unreachable or rejects the failover request	Check the origin group’s secondary origin (domain, ports, certificates) and the Host header your origin receives
Bots are not served by SpeedWorkers	Lambdas not associated, or `x-sw-*` headers not forwarded	Check both event associations on the behavior; check the origin request policy forwards `X-SW-*` headers
Every request goes to your origin	`swPrimaryOriginDomain` doesn’t match the primary origin’s domain	Set it to the exact SpeedWorkers domain configured in Step 1
Humans receive bot-rendered pages	Cache not partitioned	Add `X-SW-Request-Type` to the cache policy’s cache-key headers
403 responses served repeatedly	Error caching enabled	Set error caching minimum TTL to 0 for 403 (Step 6)
POST/PUT requests rejected with 403	Path only matched by the SpeedWorkers behavior (GET/HEAD only)	Create a dedicated behavior for that route, placed before the SpeedWorkers one
Wrong content variant served	Query strings not forwarded	Forward content-affecting query strings in the origin request policy

Lambda@Edge logs land in CloudWatch in the edge region that served the request (log groups /aws/lambda/us-east-1.<function-name>); check the regions closest to your test location.

Operational notes

Every non-bot cache miss costs two origin-request invocations and a failover round trip. This is the price of an origin-agnostic lambda. If your HTML traffic is mostly human with a low CloudFront hit ratio, the redirect variant halves those invocations and removes the extra latency.
Bot traffic intentionally bypasses the CloudFront cache. The viewer lambda gives each bot request a unique x-sw-request-type value (bot-<timestamp>-<random>), so bot responses never produce cache hits: every bot request reaches SpeedWorkers (which has its own cache) and appears in its logs and reports. The cache-key header mainly protects the user partition from bot-rendered content.
Failover is silent by design. A 403 from SpeedWorkers triggers the fallback, but so would a SpeedWorkers auth misconfiguration or outage — the site keeps working and nobody notices bots stopped getting prerendered pages. Monitor the share of requests answered by the secondary origin (x-edge-result-type in CloudFront standard logs) and your SpeedWorkers dashboards. In this variant the secondary-origin share includes all human traffic, so watch the bot partition specifically.
The SpeedWorkers domain is a hard DNS dependency for the whole behavior. CloudFront resolves the primary origin’s domain before invoking the origin-request lambda; if that resolution fails, the lambda never runs and all traffic on the behavior gets a 502.
Lambda@Edge has no environment variables. Any endpoint or API key your own additions need lives in the function code; rotating them means publishing a new version and updating the distribution (a few minutes of propagation).
Do not enable OPTIONS caching on the behavior: cached OPTIONS responses would bypass the origin-request lambda.
Alarm on the LambdaExecutionError / LambdaValidationError CloudFront metrics. The built-in try/catch forces the failover on logic errors, but runtime-level failures still surface as 5xx on the behavior’s traffic.

Deploying SpeedWorkers with CloudFront and Lambda@Edge: Force-Fail Variant