1 in 6 'Googlebots' on our sites was fake
On our five websites, about 1 in 6 requests calling itself Googlebot didn't come from Google, and more than half of the "ChatGPT" requests weren't from OpenAI. They came from rented cloud servers, looking for passwords. Here's how to tell a real crawler from a fake in a minute.
Any program can call itself anything. The user agent, the line of text where a visitor says who it is, is typed by the sender. Writing "Googlebot" in it costs nothing, and it pays: many sites wave Googlebot through rate limits, paywalls and firewalls, because nobody wants to block Google.
So between 2 and 16 October 2026 we checked every request to our five websites that claimed to be a well-known crawler against the address ranges the crawler's owner publishes.
How many were fake
| Claimed to be | Real | Fake | Share fake |
|---|---|---|---|
| ChatGPT (live fetches) | 119 | 133 | 53% |
| PerplexityBot | 411 | 122 | 23% |
| Bingbot | 446 | 124 | 22% |
| GPTBot | 532 | 143 | 21% |
| OAI-SearchBot | 567 | 154 | 21% |
| Googlebot | 1,340 | 277 | 17% |
Requests to our five sites, 2–16 Oct 2026. "Real" includes a few requests we couldn't check (they reached us through a CDN), so the fake shares are, if anything, slightly low.
Who the fakes were
Three things stood out:
- Very few senders. All 1,394 fake-crawler requests came from at most 28 addresses. A small number of tools, cycling through crawler names to see which one gets through.
- Rented servers. 98% came from data centres, and the largest share from Google's own cloud, in Taiwan, the Netherlands, Germany, Belgium, Canada and the US. Anyone can rent a server there. That is exactly why "it comes from Google's network" doesn't prove it's Googlebot.
- Not crawling at all. The fakes didn't read our articles. They asked for
/services/.env,/docker/.env,/key.json, and file paths that try to read a server's secrets through development tools. 86% got a "not found".
In other words: a crawler's name is a disguise worn for one reason, to get past defences that trust it.
Check one address in a minute: reverse DNS
Google, Microsoft and Apple give their crawlers addresses that point back to their own domains. Look up the address, then look up the name you get back, and make sure it returns the same address:
$ host 66.249.66.1
1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com.
$ host crawl-66-249-66-1.googlebot.com
crawl-66-249-66-1.googlebot.com has address 66.249.66.1Both steps matter. Anyone who controls an address can make its reverse record say googlebot.com; only Google can make googlebot.com point back to it. The names to expect:
- Googlebot: ends in
googlebot.com,google.comorgoogleusercontent.com(the last one for fetches a person triggers). - Bingbot: ends in
search.msn.com. - Applebot: ends in
applebot.apple.com.
Check at scale: the published address lists
For AI crawlers, reverse DNS usually doesn't help. Their owners publish lists of addresses instead, and Google and Microsoft do too:
- Google: googlebot.json, plus lists for special crawlers and user-triggered fetchers
- Bing: bingbot.json
- OpenAI: gptbot.json, searchbot.json, chatgpt-user.json
- Perplexity: perplexitybot.json, perplexity-user.json
- Apple: applebot.json
These lists change, so fetch them regularly instead of copying them once. If an address claims to be one of these crawlers and isn't in the matching list, it's fake.
What to do with fakes
- Never allow traffic by user agent alone. If a firewall rule, rate limit or paywall lets "Googlebot" through, check the address too. Cloudflare's "verified bot" setting does this for you.
- Block them freely. Blocking a fake Googlebot can't hurt your search rankings, because it was never Google.
- Read them as a warning. Fakes hunting for
.envand key files tell you what attackers expect to find. Make sure they can't. We cover that in our post on WordPress attacks.
Humansift runs these checks on every request automatically. Real crawlers are marked verified, impostors show up as fake crawlers, and you get an alert when a new one appears.
How we measured
Every number here comes from the server logs of our own five websites (a pet guide, an education site, a small-business tool, a crafts marketplace and a fashion blog), from 2 to 16 October 2026: 400,425 page and API requests. Images, styles and scripts were counted separately and are left out. Each request was sorted by Humansift: named bots by their user agent, Googlebot, Bingbot, GPTBot, ChatGPT and Perplexity checked against the address ranges their owners publish, attack paths by pattern, and data-centre traffic by network. A "visitor" is one address and browser on one day. Five small sites are a small sample, so read these as one honest data point, not an industry average.