Engine guide · ChatGPT

ChatGPT Search: Crawlers, Citations and What Controls Them

ChatGPT Search is the part of ChatGPT that looks up the live web, rewrites your question into targeted queries and returns an answer with links to the pages it used. For a publisher the useful question is narrow: what decides whether your page shows up in that answer, and what can you actually influence? This guide separates what OpenAI documents from what third parties claim, and says plainly where nobody outside OpenAI knows.

Diagram of ChatGPT Search showing a page passing OAI-SearchBot access, an index, query rewriting and a cited answer

What ChatGPT Search is and where it is available

OpenAI's help center describes web search as a feature that answers questions "with current information and links to relevant sources". It is available on Free, Go, Plus, Pro, Business, Enterprise and Edu plans, and people who are not signed in can use it too. Usage limits depend on the plan.

ChatGPT may decide on its own that a question needs fresh information and search the web automatically. Users can also trigger it by choosing Search in the tools menu or typing a slash in the message box. That matters for publishers because two different behaviors exist: an automatic decision by the model, and a deliberate user request. Both end in an answer that may carry citations, and a Sources view that lists cited pages and other relevant links.

OpenAI is candid about the limits of the output. Its help page warns that results and citations "can be incomplete, outdated, or incorrect" and tells users to open the cited source. Treat that as the baseline. A citation is a pointer, not a certificate.

How a ChatGPT answer is assembled

OpenAI explains one step of the pipeline directly. When ChatGPT search uses other providers, it "typically rewrites your query into one or more targeted queries" and may send further, more specific queries after reviewing the first results. Its own example turns a long question about cancer drug development into a short keyword-style query, then a second query about a specific conference.

Two consequences follow. You are rarely competing for the exact phrase the user typed, but for the rewritten sub-queries. And a page that answers a narrow sub-question clearly probably has a better chance than a broad page that mentions the topic in passing. That second point is an editorial reading, not an OpenAI statement.

How a page reaches a ChatGPT Search answer: access, index, query rewriting, citations
From your page to a ChatGPT Search citation: five stages and the three separate OpenAI agents

The help center also says ChatGPT may use approximate location from the IP address, and that saved memories can influence query rewriting when memory is enabled. So two people asking the same question can trigger different searches. Any visibility test you run has to record location and account state, or the results will not be comparable.

The OpenAI crawlers that matter

OpenAI documents four agents on its crawler page. The relationship between them is the most practical thing a webmaster can learn.

Agent Purpose Affects ChatGPT Search? robots.txt applies?
OAI-SearchBot Surfaces sites in ChatGPT search features Yes, controls inclusion Yes
GPTBot Crawls content that may train foundation models No, training only Yes
ChatGPT-User Visits pages when a user action in ChatGPT or a Custom GPT triggers it Not used to decide Search inclusion "May not apply"
OAI-AdsBot Checks landing pages submitted as ChatGPT ads No Not stated on the crawler page

OpenAI says each setting is independent. A site can allow OAI-SearchBot to appear in search results while disallowing GPTBot so its content is not used for training. If both are allowed, OpenAI says it may use one crawl for both purposes to avoid duplicate crawling.

The single most common mistake is blocking everything that starts with "GPT" or "OpenAI" and then wondering why the site never appears in ChatGPT answers. Blocking OAI-SearchBot has a direct cost, because OpenAI states that opted-out sites will not be shown in search answers, only as navigational links.

A robots.txt that allows search but declines training looks like this:

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

Allowing a bot in robots.txt is only half of access. OpenAI also recommends allowing requests from its published IP ranges, and its help page tells site owners to confirm that the host or content delivery network lets that traffic through. A firewall or bot-protection rule can silently block a crawler that robots.txt welcomes. The IP lists are published as JSON files on openai.com for each agent.

One more operational detail: OpenAI notes it can take about 24 hours after a robots.txt update for its search systems to adjust. Do not judge a change after an hour.

Which search index sits underneath

This is where sources disagree, and the disagreement is worth showing rather than hiding.

The video walkthrough reviewed for this guide states that ChatGPT search uses Bing's index, and uses that to argue for submitting a sitemap to Bing Webmaster Tools. OpenAI's help center is vaguer. It says ChatGPT search "sometimes partners with other search providers", and in the privacy section it points readers to the Microsoft and Shopify privacy statements. It does not say that every answer uses Bing, and it does not name an index for each query.

A third-party guide from Smart Money Media (a PR agency that sells related services) lists "the role of any search partner for a given answer" among the things that remain unknown. That framing is the safer one. Bing indexing is a low-cost hedge, since Microsoft appears among the providers OpenAI links to, but nobody outside OpenAI can promise that a Bing ranking becomes a ChatGPT citation. For the Microsoft side, see Microsoft Copilot search.

What OpenAI says about ranking

The only statement on ranking in the help center is short: ChatGPT "ranks search results using multiple factors intended to help users find relevant, reliable information", and "placement is not guaranteed". Eligibility, in OpenAI's words, means allowing OAI-SearchBot and confirming the host or CDN allows OpenAI's published IP addresses.

Everything beyond that is inference. Third-party guides split into two camps. Careful ones, such as the Smart Money Media guide, label items as documented, observed, plausible or unproven, and say that crawler access does not prove retrieval, retrieval does not prove selection, and a citation does not prove a visit. Others publish exact multipliers. The GeoRankers playbook (September 2026 update), for instance, reports that only 11% of domains are cited by both ChatGPT and Perplexity, that ChatGPT shows a "3.5x citation lift at 32,000+ referring domains", and that 76.4% of the most-cited ChatGPT pages were updated in the last 30 days. This guide could not trace these figures to primary studies, so they are reported, not established. GeoRankers sells a monitoring tool, which is worth remembering when reading its numbers.

Analysis of the reviewed walkthroughs

One video walkthrough was reviewed for ChatGPT: a Surfer presentation by Matt Canyon. It is an explanatory talk, not an experiment with logged prompts, so only what it says is reported here, not what it measured. All figures in it are third-party figures quoted by the presenter and are self-reported.

  • It argues that generative platforms "often pull from the top ranking search results and trusted sources", and that without solid organic rankings you are unlikely to appear.
  • It says AI systems mostly read raw HTML, so facts that appear only after a JavaScript interaction, or inside images and video, may be missed.
  • It compares Google's search volume with ChatGPT's and notes that many ChatGPT messages are not search-style queries. The presenter's figures are self-reported and are not reproduced here.
  • It mentions consumer-preference and AI Overviews studies by third parties. Those concern Google users, not ChatGPT, and their numbers are omitted because they could not be checked.
  • Its measurement advice is to track brand mentions, because conversational answers often produce no click.

The presenter's conclusion is that GEO is mostly good SEO with a few extra layers. That matches the cautious third-party guides. He also cites a 37% inclusion uplift from an unnamed 2024 study; the underlying study could not be identified, so the figure is unconfirmed.

Content patterns that tend to be cited

No public document says ChatGPT prefers lists or tables. Still, the guides reviewed converge on the same practical advice, and it costs little to follow:

  1. Open sections with a direct answer, then support it.
  2. Use real headings, short lists and tables for comparisons, because they are easy to read for people and easy to quote.
  3. Name your sources inline and link to primary ones.
  4. Keep brand facts consistent across your site and third-party profiles.
  5. Match structured data to what is visible on the page.

The closest published evidence is the GEO study (Aggarwal et al., "GEO: Generative Engine Optimization", 2024), which tested content changes such as adding statistics, quotations and citations. It reports visibility improvements of up to 40% in the paper's benchmark. That is a research benchmark of generative engines, not a ChatGPT Search test.

Claim Where it comes from Status
Opted-out sites are not shown in search answers OpenAI crawler page Documented
Placement is not guaranteed OpenAI help center Documented
ChatGPT search uses Bing's index Video walkthrough Unconfirmed by OpenAI
11% of domains are cited by both ChatGPT and Perplexity GeoRankers playbook Reported, not traced
76.4% of top-cited pages updated within 30 days GeoRankers playbook Reported, not traced
Visibility gains of up to 40% from statistics, quotes and citations Aggarwal et al., 2024 (arXiv) Published benchmark, not ChatGPT-specific

On the "update within 30 days" idea, the sensible reading is to refresh pages when their facts change, not to re-date them. The practical steps are in how to rank in ChatGPT.

Where ChatGPT Search sits among other engines

ChatGPT Search is not the only engine that builds answers from a mix of retrieval and a language model. Perplexity AI search shows numbered citations by default and runs its own crawler. Google AI Overviews depends on ordinary Google Search eligibility.

The evidence suggests one content plan mostly serves all of them, because the shared foundation is crawlable, factual, well-structured pages. Access controls and source presentation differ. The methodology page explains how claims are graded on this site.

What to do next

Start with access, because nothing else matters if OAI-SearchBot cannot fetch the page. Then make the important facts visible in HTML, write passages that answer one question each, and look for independent places that describe you accurately. Finally, measure with a fixed set of prompts. The step-by-step version is in how to rank in ChatGPT, including a prompt log template and a robots.txt checklist.

Frequently asked questions

Is ChatGPT Search the same as Bing?

No. OpenAI says ChatGPT search sometimes partners with other search providers and links Microsoft's privacy statement, but it does not say every answer uses Bing. The claim that it uses Bing's index comes from a video walkthrough, not from OpenAI's pages.

Which crawler do I allow to appear in ChatGPT answers?

OAI-SearchBot. OpenAI recommends allowing it in robots.txt and allowing its published IP ranges. GPTBot is a separate agent for training, so disallowing it does not remove you from search.

Does blocking GPTBot hide my site from ChatGPT?

Not from search, according to OpenAI. Each bot's setting is independent. Blocking GPTBot signals that your content should not be used for training foundation models.

Can ChatGPT still visit my page if I block bots?

Possibly. OpenAI says ChatGPT-User acts on user requests and that robots.txt rules "may not apply". It is not used to decide whether content appears in Search.

How long do robots.txt changes take to apply?

OpenAI says about 24 hours for its search systems to adjust after a robots.txt update.

Is there a guaranteed way to be cited?

No. OpenAI states placement is not guaranteed. Any source promising a fixed citation rate is selling certainty that the platform does not provide.

Smart Money Media's guide states that llms.txt "is not a documented ranking or citation control" for ChatGPT search. It may be a harmless experiment, but do not rely on it.

Limits of this guide

  • Verified: crawler behaviour, eligibility wording, the 24-hour robots.txt delay and the query-rewriting description (OpenAI crawler documentation and help center, read October 2026; both can change). The GEO benchmark figure is as stated in the paper's abstract.
  • Reported: the 11%, 3.5x and 76.4% figures (GeoRankers, a vendor), the Bing-index claim and the 37% uplift (video presenter, self-reported). No primary study was found for them.
  • Missing: no controlled ChatGPT prompt tests were available to this publication, and which search partner serves a given answer is unconfirmed. The only hands-on material is one explanatory video transcript.

Sources

Keep reading

Next stepHow to Rank in ChatGPT: A 6-Step Plan From OpenAI's Docs
CM

Mohamed CISSE

SEO specialist and writer at SEO for LLM. Writes about how AI answer engines retrieve and cite web pages, and labels every claim by how well it is sourced. How we research.