Skip to main content

About Resources

Updated over 11 months ago

📘 This article identifies Botify's default list of resources in LogAnalyzer and explains how to modify this list to meet your needs. LogAnalzyer, part of Botify's Analytics Suite, is available with all Botify plans.

Overview

LogAnalyzer Crawl reports provide visibility into how bots crawl your resources, which is important to protecting your crawl budget. You should minimize Google crawling resources that are unimportant for page rendering. You can use Botify’s default list of resources or define your own.

💡 Create a warning segment to identify when resources are crawled in the Overview and Resources reports.

Default Resource List

By default, Botify considers any URL with the following extensions a resource:

Default Resource List

  • .bmp

  • .css

  • .doc

  • .docx

  • .eot

  • .gif

  • .ico

  • .ief

  • .jpe

  • .jpeg

  • .jpg

  • .js

  • .less

  • .m1v

  • .mov

  • .mp2

  • .mp3

  • .mp4

  • .mpa

  • .mpe

  • .mpeg

  • .mpg

  • .ogv

  • .pbm

  • .pdf

  • .pgm

  • .png

  • .pnm

  • .ppm

  • .pps

  • .ppt

  • .ps

  • .qt

  • .ras

  • .rdf

  • .rgb

  • .rss

  • .svg

  • .swf

  • .tif

  • .tiff

  • .tsv

  • .ttf

  • .txt

  • .vcf

  • .wav

  • .webm

  • .woff

  • .woff2

  • .xbm

  • .xlm

  • .xls

  • .xml

  • .xpdl

  • .xpm

  • .xwd

Modifying the Default Resource List

If the default list of extensions does not meet your needs, define a custom list of URLs instead.

To define a custom list of resources:

  1. Navigate to the Segment Editor from the main navigation bar.

  2. With the Segments Overview article as guidance, copy and paste the following segment at the bottom of the Segments Rules Editor window:
    ​[segment:internal_static]
    @true
    or (
    path *.bmp
    path *.css
    path *.doc
    path *.docx
    path *.eot
    path *.gif
    path *.ico
    path *.ief
    path *.jpe
    path *.jpeg
    path *.jpg
    path *.js
    path *.less
    path *.m1v
    path *.mov
    path *.mp2
    path *.mp3
    path *.mp4
    path *.mpa
    path *.mpe
    path *.mpeg
    path *.mpg
    path *.ogv
    path *.pbm
    path *.pdf
    path *.pgm
    path *.png
    path *.pnm
    path *.ppm
    path *.pps
    path *.ppt
    path *.ps
    path *.qt
    path *.ras
    path *.rdf
    path *.rgb
    path *.rss
    path *.svg
    path *.swf
    path *.tif
    path *.tiff
    path *.tsv
    path *.ttf
    path *.txt
    path *.vcf
    path *.wav
    path *.webm
    path *.woff
    path *.woff2
    path *.xbm
    path *.xlm
    path *.xls
    path *.xml
    path *.xpdl
    path *.xpm
    path *.xwd
    )

  3. Edit the list to include the file types you want Botify to consider as resources.

Filtering Resources

When you have a segment defined for resources, you can filter your LogAnalyzer URL Explorer reports to include or exclude resources or add a report column to determine which URLs are defined as resources:

la_staticfilter.jpg
Did this answer your question?