AI crawler index
What is GoogleOther?
GoogleOther is a crawler run by Google. A generic crawler Google product teams use for one-off fetches, such as internal research and development.
At a glance
| Operator | |
|---|---|
| Purpose | Other or undeclared |
| User agent | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GoogleOther) Chrome/W.X.Y.Z Safari/537.36 |
| Obeys robots.txt | Yes, according to the operator |
| Published IP ranges | https://developers.google.com/static/crawling/ipranges/common-crawlers.json |
| How to verify it | Reverse DNS to crawl-*.googlebot.com or geo-crawl-*.geo.googlebot.com and forward-confirm, or match Google's IP list. |
| Official documentation | https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers |
Should you block GoogleOther?
Its purpose is not clearly documented. Check your server logs to see how often it visits and what it requests before deciding.
Google says rules for GoogleOther do not affect any specific product, Search included. GoogleOther-Image and GoogleOther-Video also exist.
How to block or allow GoogleOther in robots.txt
Block it everywhere:
User-agent: GoogleOther
Disallow: /Allow it everywhere:
User-agent: GoogleOther
Allow: /Rules are matched by the token on the User-agent line. Test the result with the robots.txt tester before you publish it.
Other Google crawlers
- Googlebot: Search engine
- Google-Extended: robots.txt control token
Check your own site
The AI crawler checker tells you which AI crawlers your robots.txt lets in, and the log file analyzer shows how often each one actually visits. For the bigger picture, see what AI crawlers are and the full AI crawler index.
FAQ
GoogleOther questions
Start tracking your brand in AI search today
Monitor how AI engines cite your brand, track keyword positions, and benchmark against competitors, all in one platform.