🛠 Botify LogAnalyzer requires information from your web server's daily logs and your website's crawl report. This document describes the methods for exporting Akamai log data to integrate with Botify.
Overview
The following table provides an overview of the export methods for Akamai logs that Botify currently supports:
Method | Process | Pros | Cons |
Akamai sends logs to Botify through FTP | Simple integration | Less secure than FTPs or SFTP | |
Botify fetches logs from Akamai through SFTP | Secure transfer protocol | More complex integration. | |
Logs are pushed to a third-party bucket and are retrieved by Botify | Simple integration | Not adapted to the largest web sites; check with Botify Support for more details. |
Logs Delivery Service (LDS)
With both LDS options, you must first activate and set up LDS. The following provides basic configuration information; please refer to the Akamai documentation for specific instructions on configuring LDS.
To activate LDS:
Set the following log configuration in your Akamai account:
(choose the "Combined+Cookie+ARL without domain+Host Header+Transfer Time+Custom Field" format).
On the Akamai property default rule, add the protocol as a custom field in the logs as shown in the screenshot below. The protocol is contained in the AK_SCHEME built-in variable.
After LDS is activated, configure the export in LDS to the Botify FTP server.
Logs Delivery Service (LDS) to NetStorage
LDS sends the logs to the Akamai Netstorage service, and then Botify securely retrieves the logs from this storage.
Follow instructions 1 and 2 above to activate LDS.
Configure the export in LDS to NetStorage as the target.
Complete the following to configure NetStorage:
Set up SSH access in the NetStorage configuration.
Provide Botify with a read-only account on NetStorage.
Configure Botify’s public key on this account.
Botify will then configure the log retrieval on NetStorage through SFTP.
DataStream 2
To configure DataStream 2 for your log exports:
Set up an AWS, Azure, or GCP bucket. If you need assistance setting up a bucket, please contact your Account Manager.
Refer to the Akamai documentation to create a stream and activate a stream, setting your destination type on the Delivery tab as your dedicated bucket where the logs will be pushed.
Ensure you identify the data set parameters identified below, which Botify requires.
Include the date in your log file name in YYYY-MM-DD format.
In Delivery options, configure the push frequency to 60 seconds.
Share the bucket credentials with Botify through a secure transfer method to allow us to pull the files when needed.
DataStream 2 Data Set Parameters
Botify requires the following data set parameters:
Log Information
CP code
Request time
Message Exchange Data
Client IP
HTTP status codes
Protocol type
Query string
Request host
Request method
Request path
User-Agent
Request Header Data
Referer
Contact Support
If you need any assistance with exporting your Akamai log data, please contact Support using the email address for your region: