meta-externalfetcher
Fetches requested by Meta AI users, for example when a user shares a link.
What happens if you block it
Meta says this fetcher may bypass robots.txt because the fetch was requested by a user. Only a firewall rule actually stops it.
Our take
Leave it allowed unless you have a reason to block user-directed fetches.
robots.txt
To block it:
User-agent: meta-externalfetcher Disallow: /
To allow it explicitly (useful when a wider rule blocks other bots):
User-agent: meta-externalfetcher Allow: /
The operator's own words
The Meta-ExternalFetcher crawler may bypass robots.txt because it performs fetches that were requested by the user.Meta documentation, checked 2026-10-31
robots.txt allows it, but is it really getting through?
A firewall or CDN rule can reject meta-externalfetcher even when robots.txt welcomes it, and nothing in your CMS shows it. Our check sends a live request as meta-externalfetcher alongside a normal browser and a Googlebot control, so a bot-verification rule isn't mistaken for a block.
Other Meta agents
Meta's crawler for use cases such as training foundation AI models or improving products by indexing content directly.
Meta's indexing crawler.
Can meta-externalfetcher reach your site?
Free, no sign-up. We request your homepage as 13 different clients and show what each one gets.