AI crawler index

What is facebookexternalhit?

facebookexternalhit is Meta's fetcher for link previews. Fetches a page or app when someone shares it on Meta's apps, to build the link preview it shows.

At a glance

OperatorMeta
PurposeLink preview
User agent facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
Obeys robots.txtNot always: the operator says some visits may ignore it
Published IP ranges Not documented by the operator
How to verify it Not documented by the operator
Official documentation https://developers.facebook.com/documentation/sharing/webmasters/web-crawlers

Should you block facebookexternalhit?

It fetches a page when someone shares its link, to build the preview card with its title, description and image. Blocking it leaves your links without a preview in those apps; it does not change AI answers or search results.

Meta says it may bypass robots.txt when performing security or integrity checks, and asks you to allowlist its user agent or IP addresses, which that page does not list. It caches what it fetches: after changing a page, refresh the preview with Meta's Sharing Debugger.

How to block or allow facebookexternalhit in robots.txt

Block it everywhere:

User-agent: facebookexternalhit
Disallow: /

Allow it everywhere:

User-agent: facebookexternalhit
Allow: /

Rules are matched by the token on the User-agent line. Test the result with the robots.txt tester before you publish it.

How to verify that a visit is really facebookexternalhit

Meta publishes no IP list or hostname for facebookexternalhit, so you cannot prove a visit is genuine. Treat the user agent as a claim and judge the visits by what they request in your logs.

Other Meta crawlers

Check your own site

The AI crawler checker tells you which AI crawlers your robots.txt lets in, and the log file analyzer shows how often each one actually visits. For the bigger picture, see what AI crawlers are and the full AI crawler index.

FAQ

facebookexternalhit questions

facebookexternalhit is Meta's fetcher for link previews. Fetches a page or app when someone shares it on Meta's apps, to build the link preview it shows.
Not always. Meta says robots.txt rules may not apply to some of its visits. A firewall or IP rule is the reliable way to stop it.
Add "User-agent: facebookexternalhit" followed by "Disallow: /" to your robots.txt. Because robots.txt may not apply to it, back the rule with a firewall or IP rule. Meta says it may bypass robots.txt when performing security or integrity checks, and asks you to allowlist its user agent or IP addresses, which that page does not list. It caches what it fetches: after changing a page, refresh the preview with Meta's Sharing Debugger.
Meta publishes no IP list or hostname for it, so only the user agent identifies it, and anyone can fake that.

Start tracking your brand in AI search today

Monitor how AI engines cite your brand, track keyword positions, and benchmark against competitors, all in one platform.