AI crawler index

What is ChatGPT-User?

ChatGPT-User is OpenAI's fetcher for pages users ask about. Visits a page when a user asks ChatGPT or a custom GPT a question that needs it, and for GPT Actions.

At a glance

Operator	OpenAI
Purpose	User-triggered fetch
User agent	`Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot`
Obeys robots.txt	Not always: the operator says some visits may ignore it
Published IP ranges	https://openai.com/chatgpt-user.json
How to verify it	OpenAI publishes its IP ranges as JSON; match the source IP against them.
Official documentation	https://developers.openai.com/api/docs/bots

Should you block ChatGPT-User?

It visits a page only when a user asks the assistant something that needs it. Blocking it stops the assistant from reading your page in those moments, which costs you citations; and because a person starts each request, robots.txt may not stop it.

Because a user starts each visit, OpenAI says robots.txt rules may not apply. It does not decide whether you appear in ChatGPT search: OAI-SearchBot does.

How to block or allow ChatGPT-User in robots.txt

Block it everywhere:

User-agent: ChatGPT-User
Disallow: /

Allow it everywhere:

User-agent: ChatGPT-User
Allow: /

Rules are matched by the token on the User-agent line. Test the result with the robots.txt tester before you publish it.

Test ChatGPT-User on your robots.txt

How to let ChatGPT-User through a firewall or CDN

robots.txt only tells ChatGPT-User what it may fetch. If a firewall, a CDN or a bot-protection rule blocks the request first, it never reads the file, so allowing it in robots.txt is not enough.

Allow the IP ranges OpenAI publishes (https://openai.com/chatgpt-user.json) rather than the user agent, which anyone can fake.

How to verify that a visit is really ChatGPT-User

Anyone can write ChatGPT-User in a user agent, so the user agent alone proves nothing. Take the IP address of the request from your server logs and check it.

OpenAI publishes its IP ranges as JSON. This command downloads the list and prints True if the IP is in it. Replace 203.0.113.7 with the IP from your logs; Python 3 comes with macOS and most Linux systems.

curl -s https://openai.com/chatgpt-user.json \
  | python3 -c 'import sys, json, ipaddress as ip
a = ip.ip_address(sys.argv[1])
ranges = json.load(sys.stdin)["prefixes"]
print(any(a in ip.ip_network(r.get("ipv4Prefix") or r.get("ipv6Prefix")) for r in ranges))' 203.0.113.7

Other OpenAI crawlers

GPTBot: AI training
OAI-SearchBot: AI search

Check your own site

The AI crawler checker tells you which AI crawlers your robots.txt lets in, and the log file analyzer shows how often each one actually visits. For the bigger picture, see what AI crawlers are and the full AI crawler index.

What our data says about ChatGPT-User

Figures we measured on real AI answers and websites. Each one links to its study, with the method, the sample and the date.

4.7%

4.7% of the websites AI cites most in Spain block ChatGPT-User from crawling their site, and 10.9% name it in a robots.txt group of its own.

Sample: 678 websites AI cites for three or more of 43 brands in Spain. Source: Mencoro (November 2026), in Which AI crawlers do the websites AI cites let in?