Traffic that is not customers.
THE SCALE
On many small sites, automated requests exceed human ones.
Each consumes resources exactly as a visitor does.
WHAT IT CONSISTS OF
Search engine crawlers, which you want in moderation Scrapers copying content Vulnerability scanners Spam bots targeting forms Aggressive crawlers with no benefit to you
HOW TO SEE IT
Raw access logs. Look at user agents and request rates.
Compare total requests against analytics.
WHAT TO DO
Caching, which serves bots cheaply from a stored copy Blocking clearly malicious agents Rate limiting A CDN or firewall filtering before requests reach you
THE CACHING POINT
A cached page served to a bot costs almost nothing.
The same page built from scratch costs the same as a real visit.
WHAT NOT TO BLOCK
Legitimate search engine crawlers.
Check any rule carefully before applying it.
WHEN TO ASK US
If bot traffic is overwhelming and you cannot control it.
