AI crawler index

What is facebookexternalhit?

facebookexternalhit is Meta's fetcher for link previews. Fetches a page or app when someone shares it on Meta's apps, to build the link preview it shows.

At a glance

OperatorMeta
PurposeLink preview
User agent facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
Obeys robots.txtNot always: the operator says some visits may ignore it
Published IP ranges Not documented by the operator
How to verify it Not documented by the operator
Official documentation https://developers.facebook.com/documentation/sharing/webmasters/web-crawlers

Should you block facebookexternalhit?

It fetches a page when someone shares its link, to build the preview card with its title, description and image. Blocking it leaves your links without a preview in those apps; it does not change AI answers or search results.

Meta says it may bypass robots.txt when performing security or integrity checks, and asks you to allowlist its user agent or IP addresses, which that page does not list. It caches what it fetches: after changing a page, refresh the preview with Meta's Sharing Debugger.

How to block or allow facebookexternalhit in robots.txt

Block it everywhere:

User-agent: facebookexternalhit
Disallow: /

Allow it everywhere:

User-agent: facebookexternalhit
Allow: /

Rules are matched by the token on the User-agent line. Test the result with the robots.txt tester before you publish it.

Test facebookexternalhit on your robots.txt

How to let facebookexternalhit through a firewall or CDN

robots.txt only tells facebookexternalhit what it may fetch. If a firewall, a CDN or a bot-protection rule blocks the request first, it never reads the file, so allowing it in robots.txt is not enough.

Meta publishes no IP list or hostname for facebookexternalhit, so the only rule you can write is on the user agent, and that also lets in anyone who copies it.

How to verify that a visit is really facebookexternalhit

Meta publishes no IP list or hostname for facebookexternalhit, so you cannot prove a visit is genuine. Treat the user agent as a claim and judge the visits by what they request in your logs.

Other Meta crawlers

Check your own site

The AI crawler checker tells you which AI crawlers your robots.txt lets in, and the log file analyzer shows how often each one actually visits. For the bigger picture, see what AI crawlers are and the full AI crawler index.

What our data says about facebookexternalhit

Figures we measured on real AI answers and websites. Each one links to its study, with the method, the sample and the date.

  • 0.7%

    0.7% of the websites AI cites most in Spain block facebookexternalhit from crawling their site, and 3.8% name it in a robots.txt group of its own.
    Sample: 678 websites AI cites for three or more of 43 brands in Spain. Source: Mencoro (November 2026), in Which AI crawlers do the websites AI cites let in?

FAQ

facebookexternalhit questions

facebookexternalhit is Meta's fetcher for link previews. Fetches a page or app when someone shares it on Meta's apps, to build the link preview it shows.
Not always. Meta says robots.txt rules may not apply to some of its visits. A firewall or IP rule is the reliable way to stop it.
Add "User-agent: facebookexternalhit" followed by "Disallow: /" to your robots.txt. Because robots.txt may not apply to it, back the rule with a firewall or IP rule. Meta says it may bypass robots.txt when performing security or integrity checks, and asks you to allowlist its user agent or IP addresses, which that page does not list. It caches what it fetches: after changing a page, refresh the preview with Meta's Sharing Debugger.
Allow it in robots.txt with "User-agent: facebookexternalhit" and "Allow: /", then make sure your firewall or CDN does not block it before it reaches the file.
Meta publishes no IP list or hostname for it, so only the user agent identifies it, and anyone can fake that.

Start tracking your brand in AI search today

Monitor how AI engines cite your brand, track keyword positions, and benchmark against competitors, all in one platform.