TDS.SO
Guides

Verify filtering

How to check that the filter lets people through and filters out bots.

To make sure everything is set up, make one visit as a person and one as a bot, then find both in the log.

Before you check

  • In the template's Bots row, choose Send to the bot URL and enter a URL — this makes the result easier to see.
  • If Treat first clicks as bots is above 0, your first visits also count as bots. Set it to 0 while you check.

Visit as a person

Open the link in a regular browser. You should land on the destination URL or the URL of the matching stream.

Visit as a bot

Send a request as a search engine crawler — it counts as a bot at any filter level:

curl -s -i -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
  "https://example.com/Ab3xZ"

For 302 Temporary and 301 Permanent, the bot URL is in the location header; for other methods, it is inside the page. If a response code is selected for bots, you get an empty page with that code.

With Double Meta Refresh, repeat the same request a second time — the first response only reloads the page.

Find the visits in the log

In Statistics → Clicks, find your visits by IP. The browser visit should be in the Humans segment, the curl request in the Bots segment.

Click log: a bot click with a red mark and a human click
Click log: a bot click with a red mark and a human click

One visit shows that the settings work, but not how good the filter is. How to choose a level based on real traffic is in Bot filtering.

If your visit was counted as a bot

  • Check Treat first clicks as bots, the bot lists and Allowed referrer: if it is filled in, a direct visit counts as a bot.
  • At the JavaScript level, a browser with JavaScript turned off counts as a bot.

On this page