Engine guide · Meta AI

Meta AI Search: Crawlers, Sources and What We Know

Meta AI search is the part of Meta's assistant that answers current questions from the open web, inside WhatsApp, Instagram, Facebook and Messenger, and it is the least documented major AI search surface a publisher can try to appear in.

Diagram of Meta AI search: Meta crawlers and web search feed a model that answers inside chat apps

What Meta AI search is, and where you meet it

Meta AI is an assistant that sits in the search bar of Meta's apps. When Meta launched the Llama 3 version on April 18, 2024, it described Meta AI as available in search across Facebook, Instagram, WhatsApp and Messenger, with access to "real-time information from across the web" (Meta newsroom). It also opened a standalone site, meta.ai.

That placement is the first thing to understand. ChatGPT, Perplexity or Google's AI features are places people go to search. Meta AI is a feature inside places people already are. Someone planning a trip in a Messenger group chat can ask it for flights without leaving the chat, which is exactly the example Meta used.

For a publisher this changes the shape of the opportunity. You are not competing for a results page. You are competing to be one of the sources behind a short answer inside a conversation. If you want the neighbouring social engine for comparison, see Grok search, which works on top of X posts that are mostly public.

Pipeline of Meta AI search from Meta crawlers and partner web search through the model to an in-app answer
How a Meta AI answer is assembled, based on Meta's crawler page and press reports

How Meta AI finds current information

Meta has never published a full retrieval diagram. What exists is a set of statements from Meta and a few journalists, and they do not fully agree.

Meta's own launch post says Meta AI can access real-time information from across the web. It does not name the search provider in its text. Search Engine Journal, writing the same day, reported that the assistant would incorporate results from both Google and Bing. The same author added a caveat: in limited testing he "could only get Meta AI to search using Bing", and he did not know what decides which engine is used.

A 2026 article from the Belgian agency Customer Impact (updated 18 October 2026) repeats the Google and Bing point, and adds that Meta's own content is a second source. Per that article, Meta says Meta AI draws recommendations from Reels, maps and public posts, with credit back to creators. The Meta statement itself was not located, so treat that part as reported.

So there are at least three layers a page might pass through:

  1. Partner web search. If Google or Bing can find and rank your page, Meta AI has something to retrieve.
  2. Meta's own crawlers. Meta runs its own bots, discussed below, and the Meta-WebIndexer entry says its purpose is improving Meta AI search quality.
  3. Meta's social content. Public posts and Reels may be used for some recommendation-style questions.

Which of these layers wins for a given question is not documented, so nobody outside Meta can say how a specific page gets chosen.

The Meta crawlers and what each one does

Meta's developer documentation, "Meta Web Crawlers", lists several agents. Pay attention to the wording, because two of them behave differently from what most robots.txt habits assume.

Crawler User-agent token Stated purpose (Meta) robots.txt
Meta-ExternalAgent meta-externalagent/1.1 Crawls the web for training foundation AI models or improving products by indexing content directly Block with a Disallow rule
Meta-ExternalFetcher meta-externalfetcher/1.1 Fetches individual links at a user's request, supports agentic AI features Meta says it may bypass robots.txt rules
Meta-WebIndexer listed on Meta's page Navigates the web to improve Meta AI search result quality Not stated in the excerpt reviewed
FacebookExternalHit link-preview agent Fetches shared links; may bypass robots.txt for security or integrity checks Partial

Two caveats. First, the Meta-WebIndexer line was seen only in a search excerpt of the French-locale rendering of Meta's page, because the page itself returned an error to the fetcher used for this guide. Second, sources disagree about FacebookBot. Business Insider described it in August 2024 as an older crawler that had scraped data for years to train Meta's language models, while a May 2026 playbook from We The Flywheel describes it as the link-preview fetcher. Check your own logs before you decide. The analytics vendor 51Degrees identifies Meta-ExternalAgent as Meta's primary AI crawler, used for AI indexing and training.

The practical point is that a Disallow rule for Meta-ExternalAgent is not a guarantee that Meta AI will never read your page. A person asking Meta AI about a URL can trigger Meta-ExternalFetcher, and Meta says that crawler may ignore robots.txt. A third-party monitor, MetaPilot, reports that Meta caches robots.txt for up to 24 hours, which would explain why a rule change does not act instantly. That figure is not from Meta.

A robots.txt example

Meta's page shows rules in this style. A selective policy, for a site that wants to stay eligible for indexing but keep a private area out, looks like this:

User-agent: meta-externalagent
Allow: /
Disallow: /private/

What Meta has not told publishers

The gaps matter more than the facts here. Judging from the sources reviewed, Meta publishes:

  • no list of ranking or citation signals for Meta AI answers;
  • no statement of which search index it uses today;
  • no report in any publisher console showing that Meta AI cited a page;
  • no guidance on how long new content takes to appear.

We The Flywheel (26 May 2026) states the same limits in its own guide, calling Meta AI the most opaque engine it covers. Another site, WP SEO AI (3 July 2025), says Meta AI uses dense retrieval and "query fan-out" and gives a 6 to 12 week timeline. None of that matches a Meta document, so it is not relied on here. The honest position is that mechanics are inferred, not confirmed.

How strong is the evidence for each source?

Possible source What it would mean for you Evidence Who says so
Google results Rank well in Google Reported, April 2024 Search Engine Journal
Bing results Rank well in Bing Reported; the only one its author could trigger Search Engine Journal
Meta-WebIndexer crawl Stay crawlable by Meta's bot Named on Meta's page, purpose stated Meta (search excerpt)
Public posts and Reels Active public social presence Reported, 2026 Customer Impact
Model training data Long-term brand mentions Plausible, not documented for answers We The Flywheel (inference)

Read this table as a map of what is claimed, not a ranking of what works. If the Bing route really carries many queries, the guide to Microsoft Copilot search and to ChatGPT Search is the closest proxy for how Bing-fed answers behave, though that is an inference and not something Meta states.

Models, regions and moving parts

The assistant has not stayed still. At the 2024 launch it ran on Llama 3. Customer Impact reports that since April 2026 the Meta AI app and website run on Muse Spark, the first model from Meta Superintelligence Labs, with a rollout to the chat apps announced. That is not verified on a Meta page, so it is flagged as reported.

Region matters too. According to Customer Impact, Meta began the European rollout on March 19, 2025, across 41 European countries and six languages, with a more limited text-focused version than in the United States. The same article says the European Commission imposed interim measures on June 9, 2026 over WhatsApp Business API access for rival assistants. That affects competition rather than citations, but it shows the landscape is still shifting.

Usage numbers are another weak spot. In September 2024 Meta said more than 400 million people used Meta AI monthly, with 185 million weekly. A May 2026 playbook says "over a billion monthly users" but gives no Meta source. Until Meta states a current figure, use the 2024 one only as a dated floor.

Measuring something you cannot see

Because there is no Meta-side report, measurement is indirect. Three approaches are reasonable:

  • Log analysis. Look for the Meta user-agents above in your server logs. This shows eligibility, not citation.
  • Manual prompts. Ask Meta AI your target questions on a schedule and record whether your domain or brand appears. Do it from more than one country if you can, because rollouts differ.
  • Brand signals. Watch branded search volume and direct visits, since a zero-click mention tends to show up there, if anywhere.

Some citation trackers say they are beginning to probe Meta AI, but We The Flywheel calls their coverage the weakest of any engine, so a tool budget is hard to justify on that basis alone. For how this site judges evidence of this kind, read the methodology.

What a publisher should do with this

The evidence suggests Meta AI is a brand-visibility surface, not a traffic channel. That is an editorial judgment, not a Meta statement. It rests on the closed-app setting and the absence of analytics.

The sensible plan is short. Keep your pages crawlable and findable in Google and Bing, since that is the best-supported route in. Decide on purpose whether Meta-ExternalAgent may crawl you, because that is a content-licensing choice. Keep facts consistent across your site and profiles. Then spend the rest of your effort where results can be measured. The step-by-step version is in how to rank in Meta AI, and the wider family of engines that share Google's index is covered in Google Gemini.

Frequently asked questions

It is Meta AI's ability to answer current questions using web information, available from the search bars of Facebook, Instagram, WhatsApp and Messenger and on meta.ai. Meta's April 2024 launch post describes access to real-time information from across the web.

Does Meta AI use Google or Bing?

Search Engine Journal reported in April 2024 that Meta AI used real-time results from both, but its author could only get Bing results in testing. Meta's own post does not name a provider. Treat the current mix as unconfirmed.

Which Meta crawlers should I know?

Meta lists Meta-ExternalAgent for training and indexing, Meta-ExternalFetcher for user-requested fetches, and Meta-WebIndexer for Meta AI search quality. FacebookExternalHit handles shared links. Check the official crawler page for the current list, because it changes.

Will blocking Meta-ExternalAgent remove me from Meta AI?

Not necessarily. Meta says Meta-ExternalFetcher fetches pages at a user's request and may bypass robots.txt. Blocking one bot affects that bot only, and other routes, such as partner search results, are unaffected.

Can I see when Meta AI cites my site?

No Meta tool that reports it was found. You can check server logs for Meta user-agents and test prompts manually. Neither proves a citation led to a visit.

Is Meta AI the same in Europe?

Not fully. Customer Impact reports a more limited, text-focused version in the EU after the March 2025 rollout. Features and languages have kept expanding, so test inside the apps your audience uses.

How many people use Meta AI?

Meta said 400 million monthly users in September 2024. A 2026 agency article claims over a billion without a source. The newer number is unconfirmed.

Limits of this guide

  • Verified: Meta's April 2024 launch post and September 2024 usage figures (Meta newsroom), the Search Engine Journal report of Google and Bing results, and the Business Insider crawler article.
  • Reported, not verified on a Meta page: the Meta-WebIndexer line (search excerpt only), the Muse Spark change, EU dates and Reels or public-posts sourcing (Customer Impact), the FacebookBot description (We The Flywheel), the robots.txt cache time (MetaPilot) and the 6 to 12 week timeline (WP SEO AI, not relied on).
  • Missing: no video walkthrough transcript exists for Meta AI, so there is no analysis section and no prompts were run for this guide. Meta publishes no ranking signals or analytics. Agency articles sell services, so their ranking claims were not used as fact. Meta changes products often, so recheck dates before acting.

Sources

Keep reading

Next stepHow to Rank in Meta AI: A Realistic 2026 Publisher Plan
CM

Mohamed CISSE

SEO specialist and writer at SEO for LLM. Writes about how AI answer engines retrieve and cite web pages, and labels every claim by how well it is sourced. How we research.