AI crawler index
What is OAI-SearchBot?
OAI-SearchBot is OpenAI's crawler for AI search. Surfaces websites in the search results of ChatGPT's search features.
At a glance
| Operator | OpenAI |
|---|---|
| Purpose | AI search |
| User agent | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot |
| Obeys robots.txt | Yes, according to the operator |
| Published IP ranges | https://openai.com/searchbot.json |
| How to verify it | OpenAI publishes its IP ranges as JSON and recommends allowing them. |
| Official documentation | https://developers.openai.com/api/docs/bots |
Should you block OAI-SearchBot?
It builds the index an AI search engine cites from. Blocking it can remove your pages from that engine's answers and links, so if you want to be cited there, allow it.
Sites that block it are not shown in ChatGPT search answers, though they can still appear as navigational links. OpenAI says robots.txt changes take about 24 hours to apply.
How to block or allow OAI-SearchBot in robots.txt
Block it everywhere:
User-agent: OAI-SearchBot
Disallow: /Allow it everywhere:
User-agent: OAI-SearchBot
Allow: /Rules are matched by the token on the User-agent line. Test the result with the robots.txt tester before you publish it.
Other OpenAI crawlers
- GPTBot: AI training
- ChatGPT-User: User-triggered fetch
Check your own site
The AI crawler checker tells you which AI crawlers your robots.txt lets in, and the log file analyzer shows how often each one actually visits. For the bigger picture, see what AI crawlers are and the full AI crawler index.
FAQ
OAI-SearchBot questions
Start tracking your brand in AI search today
Monitor how AI engines cite your brand, track keyword positions, and benchmark against competitors, all in one platform.