📘 Field descriptions for search engine bot crawls and user visits tracked via integrated web server log files.

Overview

LogAnalyzer fields are derived from your web server log files when integrated with your Botify project. They track two types of activity: search engine bot crawls (e.g., Googlebot, Bingbot) and real user visits. Together, they let you correlate how often search engines crawl your pages with how often users actually visit them.

Note: All crawl and visit fields are filtered by search engine. The default view shows Google data.

Dataset and API collection

Available in: LogAnalyzer
For API users: These fields are available in the Logs BQL collection.

Search Engine Crawls

These fields measure how often search engine bots request your pages. Identifier patterns follow crawls.google.* for Google and crawls.bing.* for Bing.

Field name	API identifier (Google)	Kind	Type	Description
Crawls: Count By Day	`crawls.google.count_by_day`	metric	integer	The number of crawls per day by the selected search engine during the current period.
Crawls: Count On Period	`crawls.google.count`	metric	integer	The total number of crawls by the selected search engine during the current period.
Crawls: Count Since Ever (end of period)	`crawls.google.count_cumulative`	metric	long	The total number of crawls by the selected search engine historically, as of the latest date in the current period.
Crawls: Count Since Ever (start of period)	`crawls.google.count_cumulative_start`	metric	long	The total number of crawls by the selected search engine historically, as of the earliest date in the current period.
Crawls: First ever seen	`crawls.google.date_first_seen`	dimension	datetime	The first time the page was ever crawled by the selected search engine, based on available log data. Datetime is expressed in the project's timezone; Date is in UTC.
Crawls: Last ever seen	`crawls.google.date_last_seen`	dimension	datetime	The most recent time the page was crawled by the selected search engine, based on available log data.
Crawls: First seen in period	`crawls.google.date_first_seen_period`	dimension	datetime	The first time the page was crawled by the selected search engine within the selected period.
Crawls: Last seen in period	`crawls.google.date_last_seen_period`	dimension	datetime	The last time the page was crawled by the selected search engine within the selected period.
Crawls: Is new	`crawls.google.is_new`	dimension	boolean	Whether the URL was crawled by the selected search engine for the first time ever during the selected period.
Crawls: Good HTTP Status Code Count By Day	`crawls.google.http_codes.ok.count_by_day`	metric	integer	The number of crawls per day where the selected search engine received a 2xx HTTP status code.
Crawls: Good HTTP Status Code Count On Period	`crawls.google.http_codes.ok.count`	metric	integer	The total number of crawls where the selected search engine received a 2xx HTTP status code during the current period.
Crawls: Bad HTTP Status Code Count By Day	`crawls.google.http_codes.bad.count_by_day`	metric	integer	The number of crawls per day where the selected search engine encountered 3xx, 4xx, or 5xx HTTP status codes.
Crawls: Bad HTTP Status Code Count On Period	`crawls.google.http_codes.bad.count`	metric	integer	The total number of crawls where the selected search engine encountered 3xx, 4xx, or 5xx HTTP status codes during the current period.

User Visits

These fields measure real user traffic to your pages as recorded in server logs. Identifier patterns follow visits.google.* for visits referred from Google and visits.bing.* for Bing.

Field name	API identifier (Google)	Kind	Type	Description
Visits: Count By Day	`visits.google.count_by_day`	metric	integer	The number of user visits per day during the current period.
Visits: Count On Period	`visits.google.count`	metric	integer	The total number of user visits during the current period.
Visits: Count Since Ever (end of period)	`visits.google.count_cumulative`	metric	long	The total number of user visits historically, as of the latest date in the current period.
Visits: Is new	`visits.google.is_new`	dimension	boolean	Whether the URL received a user visit for the first time ever during the selected period.
Visits: First ever seen	`visits.google.date_first_seen`	dimension	datetime	The first time the page was ever visited by a user, based on available log data.
Visits: Last ever seen	`visits.google.date_last_seen`	dimension	datetime	The most recent time the page was visited by a user, based on available log data.
Visits: Good HTTP Status Code Count By Day	`visits.google.http_codes.good.count_by_day`	metric	integer	The number of pages per day that returned 2xx HTTP status codes to users.
Visits: Bad HTTP Status Code Count By Day	`visits.google.http_codes.count_bad_by_day`	metric	integer	The number of pages per day that returned 3xx, 4xx, or 5xx HTTP status codes to users.
Visits: Age between first crawl and visit (by day)	`visits.google.ages.visited_crawl_first_seen.age_by_day`	metric	integer	The number of days between the first time the URL was crawled by the search engine and the first user visit.
Visits: Age between last crawl and visit (by day)	`visits.google.ages.visited_crawl_last_seen.age_by_day`	metric	integer	The number of days between the last time the URL was crawled by the search engine and the last user visit.
Visits: Average age between first crawl and visit	`visits.google.ages.visited_crawl_first_seen.age_avg`	metric	float	The average number of days between the first time the URL was crawled and the first user visit, across the current period.
Visits: Average age between last crawl and visit	`visits.google.ages.visited_crawl_last_seen.age_avg`	metric	float	The average number of days between the last time the URL was crawled and the last user visit, across the current period.

See also:

Crawl and Visit Fields (Logs)

Overview

Dataset and API collection

Search Engine Crawls

User Visits