Knowledgebase

Bot Traffic and How to Handle It Print

  • cloudflarecdn, cloudflare, spam, resourcelimits, billing, guide, howto, solution
  • 0

Automated visitors, wanted and unwanted.

WHAT BOTS ARE

Automated clients. Some are useful: search engine crawlers, monitoring services, link previews.

Many are not: scrapers, vulnerability scanners, spam tools.

WHY IT MATTERS

On many small sites, bot traffic exceeds human traffic and consumes a meaningful share of resources.

That shows up as resource limits reached for no apparent reason.

WHAT CLOUDFLARE PROVIDES

Bot fight mode on the free plan, which challenges traffic identified as automated.

More sophisticated management on paid plans.

WHAT TO ALLOW

Search engine crawlers, or you disappear from search Monitoring services you use Payment gateway callbacks, which are automated and essential

Blocking a gateway's webhook breaks your orders.

WHAT TO BLOCK

Vulnerability scanners Aggressive scrapers Anything generating volume with no benefit to you

CHECKING THE EFFECT

Cloudflare analytics separates automated from human traffic. The proportion is frequently surprising.

BEFORE BLOCKING BROADLY

Confirm you are not blocking something you depend on.


Was this answer helpful?
Back

Are you happy with your experience? Leave us a review on Trustpilot.


Trustpilot