AI crawler index

What is PerplexityBot?

PerplexityBot is Perplexity's crawler for AI search. Surfaces and links websites in Perplexity's search results. Perplexity says it is not used to train foundation models.

At a glance

Operator	Perplexity
Purpose	AI search
User agent	`Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)`
Obeys robots.txt	Yes, according to the operator
Published IP ranges	https://www.perplexity.com/perplexitybot.json
How to verify it	Match both the user agent and the source IP against Perplexity's published list.
Official documentation	https://docs.perplexity.ai/docs/resources/perplexity-crawlers

Should you block PerplexityBot?

It builds the index an AI search engine cites from. Blocking it can remove your pages from that engine's answers and links, so if you want to be cited there, allow it.

Blocking it removes your site from Perplexity's search results, and Perplexity recommends allowing it. Changes can take up to 24 hours.

How to block or allow PerplexityBot in robots.txt

Block it everywhere:

User-agent: PerplexityBot
Disallow: /

Allow it everywhere:

User-agent: PerplexityBot
Allow: /

Rules are matched by the token on the User-agent line. Test the result with the robots.txt tester before you publish it.

Test PerplexityBot on your robots.txt

How to let PerplexityBot through a firewall or CDN

robots.txt only tells PerplexityBot what it may fetch. If a firewall, a CDN or a bot-protection rule blocks the request first, it never reads the file, so allowing it in robots.txt is not enough.

Allow the IP ranges Perplexity publishes (https://www.perplexity.com/perplexitybot.json) rather than the user agent, which anyone can fake.

How to verify that a visit is really PerplexityBot

Anyone can write PerplexityBot in a user agent, so the user agent alone proves nothing. Take the IP address of the request from your server logs and check it.

Perplexity publishes its IP ranges as JSON. This command downloads the list and prints True if the IP is in it. Replace 203.0.113.7 with the IP from your logs; Python 3 comes with macOS and most Linux systems.

curl -s https://www.perplexity.com/perplexitybot.json \
  | python3 -c 'import sys, json, ipaddress as ip
a = ip.ip_address(sys.argv[1])
ranges = json.load(sys.stdin)["prefixes"]
print(any(a in ip.ip_network(r.get("ipv4Prefix") or r.get("ipv6Prefix")) for r in ranges))' 203.0.113.7

Other Perplexity crawlers

Perplexity-User: User-triggered fetch

Check your own site

The AI crawler checker tells you which AI crawlers your robots.txt lets in, and the log file analyzer shows how often each one actually visits. For the bigger picture, see what AI crawlers are and the full AI crawler index.

What our data says about PerplexityBot

Figures we measured on real AI answers and websites. Each one links to its study, with the method, the sample and the date.

5.8%

5.8% of the websites AI cites most in Spain block PerplexityBot from crawling their site, and 13% name it in a robots.txt group of its own.

Sample: 678 websites AI cites for three or more of 43 brands in Spain. Source: Mencoro (November 2026), in Which AI crawlers do the websites AI cites let in?