Skip to main content

JavaScript Crawl Fields

Field descriptions for Botify's JavaScript rendering analysis, which identifies content, links, and metadata that depend on client-side rendering.

Overview

JavaScript Crawl fields compare the raw HTML response (pre-JS) with the fully rendered DOM (post-JS). Botify's crawler executes JavaScript and captures the difference, letting you identify content, links, and metadata that are only visible after client-side rendering. This is essential for diagnosing JS-dependent SEO issues such as invisible internal links, missing metadata, or content hidden from Googlebot's first render.

Dataset and API collection

  • Available in: SiteCrawler

  • For API users: These fields are available in the Crawl BQL collection.

Rendering

Field name

API identifier

Description

Emulated Device

js.rendering.device

The device type emulated during JavaScript rendering.

JS Rendering Attempted

js.rendering.exec

Whether JavaScript rendering was attempted for this URL.

JS Rendering Successful

js.rendering.ok

Whether JavaScript rendering completed successfully.

JS Rendering Status

js.rendering.status

The status code or outcome of the JavaScript rendering process.

JS Render Time

js.rendering.time_ms

Time taken to render the page with JavaScript, in milliseconds.

Total Crawl and Render Time

total_crawl_and_render_time

The total time to both crawl and render the page, in milliseconds.

Performance Timers

Field name

API identifier

Description

First Paint (P1)

js.timers.fp_ms

Time to first paint, in milliseconds.

First Image Paint

js.timers.fip_ms

Time to first image paint, in milliseconds.

First Contentful Paint (P2)

js.timers.fcp_ms

Time to first contentful paint, in milliseconds.

First Meaningful Paint (P3)

js.timers.fmp_ms

Time to first meaningful paint, in milliseconds.

First Text Paint

js.timers.ftp_ms

Time to first text paint, in milliseconds.

Event DOM Content Received

js.timers.dom_ms

Time at which the DOMContentLoaded event fired, in milliseconds.

Event Load Received

js.timers.loaded_ms

Time at which the page load event fired, in milliseconds.

Resources — Summary

Field name

API identifier

Description

Size of Resources Executed

js.resources.size.total

Total size of all resources executed during rendering, in bytes.

No. of Resources Found

js.resources.nb.all

Total number of resources discovered during JavaScript rendering.

No. of Resources Allowed

js.resources.nb.allowed

Number of resources not blocked by any rule.

No. of Resources Blocked by Robots.txt

js.resources.nb.blocked_robotstxt

Number of resources blocked because they are disallowed by robots.txt.

No. of Resources Failed to Fetch

js.resources.nb.fetch_err

Number of resources that returned an error when fetched.

No. of Resources Executed

js.resources.nb.used

Number of resources actually executed during rendering.

No. of Resources Not Cacheable

js.resources.nb.not_cacheable

Number of resources with no-cache or non-cacheable headers.

No. of Calls that Returned a JSON

js.resources.nb.json

Number of resource calls that returned a JSON response.

No. of XHR Calls

js.resources.nb.xhr

Number of XMLHttpRequest calls made during rendering.

No. of Websocket Calls

js.resources.nb.websocket

Number of WebSocket connections initiated during rendering.

No. of Images Found

js.resources.nb.images

Number of image resources discovered during rendering.

No. of CSS Found

js.resources.nb.css

Number of CSS resources discovered during rendering.

No. of Fonts Found

js.resources.nb.fonts

Number of font resources discovered during rendering.

No. of Media Found

js.resources.nb.medias

Number of media resources (video, audio) discovered during rendering.

Resources — Detail

Field name

API identifier

Description

Resource HTTP Verb

js.resources.http_verb

The HTTP method used to fetch the resource (e.g., GET, POST).

Resource URL

js.resources.url

The URL of the resource fetched during rendering.

Resource HTTP Code

js.resources.http_code

The HTTP status code returned for the resource.

Resource Byte Size

js.resources.byte_size

The size of the resource response, in bytes.

Resource Content Type

js.resources.content_type

The MIME type of the resource (e.g., application/javascript, text/css).


See also:

Did this answer your question?