📘 Botify LogAnalyzer requires information from your web server's daily logs and your website's crawl report. This document describes the integration process when Fastly hosts the logs.
Overview
Botify supports the following methods of integrating Fastly logs:
SFTP: Fastly sends logs to Botify via Secure File Transfer Protocol (SFTP).
AWS S3 Export: Fastly sends logs to an AWS S3 bucket, and Botify fetches the logs.
SFTP
You provide us with a public SSH key to enable us to fetch your logs by SFTP. Please refer to the SFTP reference documentation from Fastly for information on setting up the configuration.
Setting up SFTP includes the following steps:
You create an SSH key.
Botify creates an SFTP folder and shares the URL with you.
Creating an SSH key
To create an SSH key for SFTP, type the following to generate the key pair on your client machine:
bash ssh-keygen -t ed25519 -f ~/.ssh/sftp_key -C "your-email@example.com"
👀 You will be prompted for a passphrase; this is optional, but recommended.
This creates two files:
The private key you will enter in Fastly as the secret key:
~/.ssh/sftp_keyThe public key you will send to Botify:
~/.ssh/sftp_key.pub
Completing the Fastly configuration
You will identify the folder URL shared by Botify where you will enter the secret key in Fastly's settings.
URL format:
project_name.upload.botify.comComplete path to upload the files:
project_name.upload.botify.com/logsCompression format = Gzip
AWS S3
Please refer to Fastly's AWS S3 reference documentation for information on setting up the configuration. The following credentials are required:
AWS Access Key Id
AWS Secret Access Key
Bucket name
Path
Domain
The following are the recommended settings when you configure Fastly exports to AWS S3:
Log format: %t %v %h %l %u "%r" %>s %b "%{Referer}i" "%{User-agent}i"
Timestamp format: %Y-%m-%dT%H:%M:%S.000
Bucket name, Access Key, Secret Key: fill in the information provided by Botify
Period: 3600
Compression: Gzip
Contact Support
If you need any assistance, please contact Support.