Meta read this site 44,648 times.
It has never sent a visitor.
Over about 20 hours, Meta’s training crawler requested 1,576 pages of limitsregistry.com — roughly 28 times each. We only know because we were watching. Nothing in ordinary analytics would have shown it, and nothing in the server bill would have named it.
limitsregistry.com is ours. We are not borrowing a customer’s traffic to make a point.
The measurement
Every AI crawler that has read one small website
All time, by crawler, as recorded. Training crawlers are the ones that can never send a reader back — that is the category distinction worth knowing.
| Crawler | Operator | What it does | Requests |
|---|---|---|---|
| Meta-ExternalAgent | Meta | Trains models. Never sends visitors. | 44,648 |
| OAI-SearchBot | OpenAI | Indexes for an AI search product. Can send visitors. | 23,478 |
| GPTBot | OpenAI | Trains models. Never sends visitors. | 17,160 |
| Amazonbot | Amazon | Indexes for an AI search product. Can send visitors. | 6,576 |
| PerplexityBot | Perplexity | Indexes for an AI search product. Can send visitors. | 3,240 |
| ChatGPT-User | OpenAI | Fetches a page for someone asking a question. Can send visitors. | 686 |
| Applebot | Apple | Indexes for an AI search product. Can send visitors. | 221 |
| DuckAssistBot | DuckDuckGo | Fetches a page for someone asking a question. Can send visitors. | 44 |
| Claude-User | Anthropic | Fetches a page for someone asking a question. Can send visitors. | 8 |
| ClaudeBot | Anthropic | Trains models. Never sends visitors. | 1 |
| Total AI crawler requests | 96,062 | ||
What this does and does not show
The honest reading
A single site, published whole, with the parts that argue against us left in.
This is one small website. Over the same period it received 29 visits from Google. Do not read 96,062 requests as evidence of scale — read it as the ratio between what was taken and what came back.
Not every crawler is the same. OAI-SearchBot, Applebot and Amazonbot index for search products that genuinely can return readers. Blocking those costs you something. Meta-ExternalAgent and GPTBot train models, and no amount of patience turns them into visitors.
Charging is not a switch we control. VisitorPing never takes a payment from a crawler. Pricing is published in a licence and collected through Cloudflare’s Pay Per Crawl, which is in closed beta and which the crawler must also join. A crawler that has not joined is refused, not billed.
We found this the hard way. The crawl pushed our own database usage from roughly 1,700 queries an hour to 29,000 and triggered a warning from our host. The feature exists because the surprise was expensive.
What you can do about it
See them, choose, and set your terms
Find out who is reading your website.
Add your site, install the tracker, and see the crawlers by name.
No credit card requiredFigures measured 2026-10-03 on limitsregistry.com