How to allow, verify or block Meta AI fetcher (meta-externalfetcher)
Leave the meta-externalfetcher token unblocked at your firewall.
Checked October 20, 2026. This covers Meta AI link fetches, not the Muse agent, which Meta does not document for site owners.
How it identifies itself
| Signal | Value | Documented by the maker |
|---|---|---|
| user-agent | meta-externalfetcher/1.1 | Yes developers.facebook.com |
- Allow it
- Leave the meta-externalfetcher token unblocked at your firewall.
- Verify it
- Match the meta-externalfetcher/1.1 user agent. Meta's crawler docs are the reference, and we have no IP list for it in our records.
- Block it
- Block the token at your firewall. Meta says this user-requested fetcher may bypass robots.txt, so robots.txt is advisory for it.
robots.txt lines for Meta AI fetcher (meta-externalfetcher)
Copy the group you want for meta-externalfetcher. The most specific matching group wins over your * group.
Let it in
User-agent: meta-externalfetcher
Allow: /Keep it out
User-agent: meta-externalfetcher
Disallow: /What we can and can't confirm
Every signal above is published by the maker. Even so, a user agent string is easy to fake, so check requests against the maker's IP list or signature where one exists before you treat them as genuine.