Claude web search is the feature that lets Anthropic's assistant look things up online and answer with cited sources, and it behaves differently from the engines most SEOs know.
What Claude web search actually is
Claude does not carry a live copy of the internet in its head. For questions about recent events, prices or specific companies, it calls a search tool, reads what comes back, and writes an answer that points to its sources. Anthropic's developer documentation describes the loop in three steps: Claude decides whether to search, the platform runs the searches (possibly several in one request), and Claude finishes with a response that cites what it used.
That sounds like every other answer engine. The difference is how much is visible. Claude shows the query it ran and the pages it pulled, which makes it one of the easier assistants to test by hand. If you want to know whether your page is in play, you can simply ask the question your buyer would ask and read the citations.
For site owners the useful question is not "how do I rank?" but "which door does my page need to walk through?" The rest of this guide maps those doors: crawlers, index, retrieval and citation. When you are ready to act, the companion guide on how to rank in Claude turns this into a checklist.
The three Anthropic crawlers
Anthropic's help center page on crawling (updated April 7, 2026) lists three robots and says they exist "to enable website owner transparency and choice." They are not interchangeable.
| Bot | What it does | If you block it |
|---|---|---|
| ClaudeBot | Collects public web content that could contribute to model training | Your future content is excluded from training datasets |
| Claude-User | Fetches a page when a person asks Claude something | Claude cannot retrieve your page for that user's query; visibility for user-directed search may drop |
| Claude-SearchBot | Crawls to improve search result quality | Your content is not indexed for search optimisation; visibility and accuracy in results may drop |
Two points trip people up. First, a blanket block on "Claude" in a hurry, often aimed at training, also shuts off the two bots that matter for citations. Second, the bots read robots.txt, and Anthropic says blocking by IP address may not work, because it stops the company from reading your robots.txt in the first place. Anthropic publishes the IP ranges it uses at claude.com/crawling/bots.json, so you can verify a suspicious hit.
Here is the opt-out syntax Anthropic itself shows, which blocks only the training crawler:
User-agent: ClaudeBot
Disallow: /
Do this per subdomain. If you also want to slow the crawler rather than stop it, Anthropic supports the non-standard Crawl-delay directive.
Where the results come from
This is the part people argue about. On March 19, 2025, Anthropic's subprocessor list gained a new entry: Brave Search. Simon Willison confirmed the link in his own test: asking Claude to search for pelican facts returned ten citations that exactly matched the same search on Brave. He also noticed that the tool definition Claude revealed had a parameter type called BraveSearchParams. TechCrunch reported the same evidence, with the careful wording that it "appears" Brave powers the feature.
A search-visibility vendor, Scrunch, added two observations on launch day. Results lined up closely with Brave's top organic results, which suggests little re-ranking. And there was no sign of a bot fetching the page live at the moment of citation, so Claude seemed to work from stored page content.
Now the complication. A third-party tracker of vendor lists shows Anthropic subprocessors for "Web Search" that include Brave Search, Parallel Web Systems and TurboPuffer, with entries dated September and October 2026. That listing was not cross-checked against Anthropic's own page, so treat it as reported. The cautious reading is that Brave remains part of the stack, but nobody should assume it is the only source.
Brave and Google are different indexes
Brave has its own index. It is not Google's index with a different logo. Scrunch notes that Brave offers no robots.txt token for its indexing bot; it follows the rules you set for Googlebot and respects noindex on individual pages. That has a practical meaning: a page you hid from Google for crawl-budget reasons is probably hidden from Brave too. For a deeper look at that engine, see our Brave Search AI overview.
How Claude decides to search
According to Anthropic's documentation, Claude searches when a request depends on information that is current, changing or outside its training data. The list includes recent events, current prices and statistics, and facts about specific organisations or products that might have changed. It answers directly for stable knowledge such as math, coding concepts, creative writing, and analysis of text already in the conversation.
Developers can steer this through a system prompt and cap it with max_uses. Anthropic says simple factual queries typically use one to three searches, while comparative research can use ten or more. That last detail matters for publishers. A comparison question like "alternatives to tool X" can trigger many searches, which widens the number of pages that get a chance to be read.
On cost, Anthropic prices web search on its API at $10 per 1,000 searches plus token costs. That is a developer detail, but it hints at why the assistant may skip search when it thinks it already knows the answer.
How citations are built
In the API, citations are always enabled for web search. Each one includes the page URL, the page title, and a cited_text field of up to 150 characters. Results also carry a page_age value, described as when the site was last updated.
Think about what that implies. The model quotes a snippet, not your whole page. Short, self-contained passages that answer a question outright are easier to quote than a long preamble. Anthropic also says that anyone displaying API outputs to end users must include citations to the original source, so the link back to you is part of the product design.
A worked example of how this plays out. Imagine a buyer asks, "Which note-taking app handles offline sync best?" Claude searches, retrieves ten results, and then cites two or three. A page that opens with "App X syncs offline by queuing edits locally and merging on reconnect" hands the model a quotable sentence. A page that opens with a company history does not.
Analysis of the reviewed walkthroughs
Two video walkthrough transcripts were reviewed for this guide. One covers using Claude as an SEO assistant by feeding it Google Search Console data. It is not about being cited by Claude, so it is background only. Its performance figures are self-reported by a presenter who promotes a paid community, and they differ from figures in other videos, so none are quoted here.
The second transcript is directly relevant. The presenter says Claude uses Brave as its search engine and that Brave's index is small compared with Google's, which he frames as less competition. He quotes an overlap figure of 87% between Claude's citations and Brave's top results, plus examples where top-five results matched fully for queries about running shoes and dog food. He also says overlap with Google was far lower, in some queries zero. These numbers come from another article shown in the video, so they are second-hand and unverified.
His practical advice: submitting your URL to Brave makes you eligible, not cited. He then lists three things the AI is looking for: a liftable answer near the top of the page, one clear signal, and trusted mentions. He also suggests asking Claude buyer questions, collecting the pages it cites, and trying to get listed on them. All of this is opinion from a single presenter, but it matches the documented mechanics above. Neither walkthrough shows a controlled test with before-and-after citation counts.
Claude compared with neighbouring engines
| Engine | Search backend (reported) | Page fetch behaviour | Citation style |
|---|---|---|---|
| Claude | Brave Search, plus other subprocessors listed by a tracker | Three documented Anthropic bots | URL, title, up to 150 characters of quoted text |
| ChatGPT | Closely tied to Bing results (Omnius claims up to 87%+ of SearchGPT citations match Bing's top results) | Not covered by the sources reviewed here | See the ChatGPT Search overview |
| Perplexity | Described as independent in the site's Perplexity guide | Not covered by the sources reviewed here | See the Perplexity AI search overview |
| Mistral Le Chat | Described as independent in the site's Mistral guide | Not covered by the sources reviewed here | See the Mistral Le Chat overview |
The Omnius figure for ChatGPT is a vendor claim about Bing overlap and should be read that way. The point of the table is the pattern: engines differ in who owns the index, which is why advice that works for one often fails for another. The methodology page explains how such claims are weighed.
What you can and cannot know
You can know which robots Anthropic runs, what each does, and how to allow or block them. You can know that Claude cites with URL, title and a short quote. You can inspect a result yourself by asking Claude a question and reading the sources.
You cannot know how sources are ranked once retrieved. Anthropic has not published ranking factors. Agencies claim Claude favours neutral, documentation-style writing. Infrasity's February 2026 post says this, and also that Claude filters content through safety checks before relevance. Nothing in Anthropic's documentation supports the second claim, so it should not anchor a strategy. What holds up is plain: be crawlable, be indexed by the backend, and write passages worth quoting. The step-by-step version is in our guide to ranking in Claude.
Frequently asked questions
Does Claude web search use Google?
There is no evidence that it does. Anthropic added Brave Search as a subprocessor in March 2025, and independent tests matched Brave's results closely. A tracker also lists other web-search subprocessors in late 2026, so the exact mix is not fully confirmed.
Which Anthropic bot do I need to allow to be cited?
Claude-SearchBot and Claude-User are the two tied to search and to user questions. ClaudeBot relates to training. Anthropic says disabling the first two may reduce your visibility in search results.
Does blocking ClaudeBot remove me from Claude answers?
Per Anthropic, it signals that your future content should be excluded from training datasets. It does not, by its own description, control search retrieval. That depends on the other two bots and on the search backend.
Does Claude cite every page it reads?
No. A retrieved result is not automatically cited: Anthropic's documentation describes Claude choosing what to cite from the results returned, and Anthropic does not publish how that choice is made.
Can I see how often Claude visits my site?
Your server logs will show the user agents named in Anthropic's help page. Anthropic also publishes its IP ranges so you can confirm a request is genuine.
Is there a way to submit my site to Claude?
Not directly. If the backend is Brave, the reported route is to make sure Brave can index your pages. A transcript presenter says submitting a URL to Brave makes you eligible, not cited; the submission page itself was not verified for this guide.
Does Claude web search work the same in the app and the API?
The API documentation describes the tool, its citations and its controls. Consumer behaviour may differ, and this guide did not test every plan or region.
Limits of this guide
- Verified: the three Anthropic crawlers and their effects (Claude Help Center, April 7, 2026); citation fields and search triggers (Anthropic developer documentation); the March 19, 2025 Brave subprocessor entry (via Simon Willison).
- Reported: the Brave link beyond that entry (independent tests by Simon Willison, TechCrunch and Scrunch, March 2025); the extra web-search subprocessors (a third-party tracker, not cross-checked against Anthropic's page); the 87% Bing match for ChatGPT (Omnius); the Infrasity claims on tone and safety filtering.
- Unverified: the 87% Claude-to-Brave overlap and the query-level overlap figures, which are second-hand in one video.
- Missing: Anthropic does not publish how Claude ranks or selects sources. Two video transcripts were reviewed, one of them off-topic for citations, so the walkthrough analysis is thin. This guide ran no original query tests, and the consumer app and regional behaviour were not checked.
Sources
- Does Anthropic crawl data from the web, and how can site owners block the crawler? (Claude Help Center, April 7, 2026)
- Web search tool (Anthropic developer documentation)
- Anthropic Trust Center: Brave Search added as a subprocessor (Simon Willison's Weblog, March 21, 2025)
- Anthropic appears to be using Brave to power web search for its Claude chatbot (TechCrunch, March 21, 2025)
- Anthropic has released web search for Claude (Scrunch, March 21, 2025)
- Anthropic subprocessor listing (thirdpartyindex.com, third-party compilation, entries dated September and October 2026)
- 6 Best GEO Practices For Ranking in AI Search Engines (Omnius, updated August 2026; source of the Bing overlap claim; page blocks automated fetching, so the URL could not be re-verified)
- How to Rank in Claude: Best Practices & Tips (2026) (Infrasity, February 8, 2026)
- Video walkthrough transcript, "Claude AI SEO, zero to 200 clicks a day" (third-party creator, reviewed 2026; original URL not found)
- Video walkthrough transcript, Claude uses Brave as its search engine (third-party creator, referencing a Chris Long LinkedIn post, reviewed 2026; original URL not found)