AI crawler index

What is Amzn-SearchBot?

Amzn-SearchBot is Amazon's search engine crawler. Improves search experiences in Amazon products and services. Amazon says it does not crawl content for generative AI model training.

At a glance

Operator	Amazon
Purpose	Search engine
User agent	`Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amzn-SearchBot/0.1) Chrome/W.X.Y.Z Safari/537.36`
Obeys robots.txt	Yes, according to the operator
Published IP ranges	https://developer.amazon.com/amazonbot/searchbot-ip-addresses/
How to verify it	Amazon publishes the IP addresses it crawls from; match the source IP against them.
Official documentation	https://developer.amazon.com/amazonbot

Should you block Amzn-SearchBot?

It feeds search in Amazon's products and services, and Amazon says it does not collect training data. Blocking it can keep your pages out of those results, so allow it unless you have a reason not to.

If robots.txt does not name Amzn-SearchBot but allows other search bots, it follows their rules. It ignores Crawl-delay and may use a robots.txt cached within the last 30 days.

How to block or allow Amzn-SearchBot in robots.txt

Block it everywhere:

User-agent: Amzn-SearchBot
Disallow: /

Allow it everywhere:

User-agent: Amzn-SearchBot
Allow: /

Rules are matched by the token on the User-agent line. Test the result with the robots.txt tester before you publish it.

How to verify that a visit is really Amzn-SearchBot

Anyone can write Amzn-SearchBot in a user agent, so the user agent alone proves nothing. Take the IP address of the request from your server logs and check it.

Amazon lists the IP addresses it crawls from on its own page: the IP of the request must be on it.

Other Amazon crawlers

Amazonbot: AI training

Check your own site

The AI crawler checker tells you which AI crawlers your robots.txt lets in, and the log file analyzer shows how often each one actually visits. For the bigger picture, see what AI crawlers are and the full AI crawler index.