AI crawler index

What is OAI-SearchBot?

OAI-SearchBot is OpenAI's crawler for AI search. Surfaces websites in the search results of ChatGPT's search features.

At a glance

Operator	OpenAI
Purpose	AI search
User agent	`Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot`
Obeys robots.txt	Yes, according to the operator
Published IP ranges	https://openai.com/searchbot.json
How to verify it	OpenAI publishes its IP ranges as JSON and recommends allowing them.
Official documentation	https://developers.openai.com/api/docs/bots

Should you block OAI-SearchBot?

It builds the index an AI search engine cites from. Blocking it can remove your pages from that engine's answers and links, so if you want to be cited there, allow it.

Sites that block it are not shown in ChatGPT search answers, though they can still appear as navigational links. OpenAI says robots.txt changes take about 24 hours to apply.

How to block or allow OAI-SearchBot in robots.txt

Block it everywhere:

User-agent: OAI-SearchBot
Disallow: /

Allow it everywhere:

User-agent: OAI-SearchBot
Allow: /

Rules are matched by the token on the User-agent line. Test the result with the robots.txt tester before you publish it.

How to verify that a visit is really OAI-SearchBot

Anyone can write OAI-SearchBot in a user agent, so the user agent alone proves nothing. Take the IP address of the request from your server logs and check it.

OpenAI publishes its IP ranges as JSON. This command downloads the list and prints True if the IP is in it. Replace 203.0.113.7 with the IP from your logs; Python 3 comes with macOS and most Linux systems.

curl -s https://openai.com/searchbot.json \
  | python3 -c 'import sys, json, ipaddress as ip
a = ip.ip_address(sys.argv[1])
ranges = json.load(sys.stdin)["prefixes"]
print(any(a in ip.ip_network(r.get("ipv4Prefix") or r.get("ipv6Prefix")) for r in ranges))' 203.0.113.7

Other OpenAI crawlers

GPTBot: AI training
ChatGPT-User: User-triggered fetch

Check your own site

The AI crawler checker tells you which AI crawlers your robots.txt lets in, and the log file analyzer shows how often each one actually visits. For the bigger picture, see what AI crawlers are and the full AI crawler index.

What our data says about OAI-SearchBot

Figures we measured on real AI answers and websites. Each one links to its study, with the method, the sample and the date.

6.5%

6.5% of the websites AI cites most in Spain block OAI-SearchBot from crawling their site, and 12.6% name it in a robots.txt group of its own.

Sample: 382 websites AI cites for three or more of 43 brands in Spain. Source: Mencoro (October 2026), in Which AI crawlers do the websites AI cites let in?