GrowthHasten

How to Rank on ChatGPT: What OpenAI's Crawler Docs Say

ChatGPT's retrieval layer is usually described as Bing's index. OpenAI's own crawler documentation names four bots and no index at all. Here is what is actually documented: which token governs citation, and whether per-engine work earns its place.

Anshuman Sinha

Written by Anshuman Sinha

Published September 6, 2026
Updated September 6, 2026
15 min read
Close-up of a hand flipping through a stack of paper documents

ChatGPT does not rank pages. It retrieves a handful of them, writes an answer over the top, and links the ones it used, so there is no position to climb and no ranked list to enter. This guide is for growth and marketing leads at B2B and SaaS companies who have watched a competitor get named in an answer and want to know what the engine is actually doing before they budget for it. One clarification first, because two different questions share the same phrasing: "how to rank on ChatGPT" here means getting your website cited inside ChatGPT's answers, not asking ChatGPT to rank or score something for you. What follows covers the four crawlers OpenAI documents, which one governs citation eligibility, how a browsed answer differs from one written without browsing, and whether any of this earns a separate line in your plan.

The short version

  • OpenAI's crawler documentation names four user agents and does not mention Bing, Microsoft, or any underlying search index. The Bing claim is repeated constantly and, in the pages we opened, never carries a source.
  • Four crawlers exist and only one of them is documented as deciding whether you can be cited. It is OAI-SearchBot, and it is rarely the token an existing robots file was written for.
  • There is no position one to win here, which means the metric you report is coverage of a question, not a rank.
  • Two different routes reach your page, and OpenAI documents that robots.txt governs them differently. Whether an answer browsed at all is decided before any of your work matters.
  • For most B2B teams the ChatGPT-specific work is a configuration check and a measurement habit. What actually earns the citation is cross-engine work already on your roadmap.

Does ChatGPT use Bing's index?

OpenAI's crawler documentation does not say, and that is a sharper answer than the one this question usually gets. On September 6, 2026 I read OpenAI's crawler documentation end to end. It names four user agents, states what each one is for, explains how robots.txt governs them, and links the published IP ranges for each. It does not mention Bing. It does not mention Microsoft. It names no underlying search index at all.

Read the limit of that carefully, because the limit is the whole value of the finding. "OpenAI's crawler documentation does not name an underlying index" is a claim about what a specific, dated document contains, and you can check it in about ten minutes. "ChatGPT does not use Bing" is a claim about a product, and nothing here supports it. The two sentences look almost identical and only one of them is verifiable.

The verifiable version is the narrower one, and it still changes what you do.

  • The retrieval layer is not a surface you configure: whatever sits behind the search feature, the documented interface between your website and it is a user-agent token and your robots.txt file. That is the part you can act on, and it is the same regardless of what the index turns out to be.
  • No third-party submission path is documented: OpenAI's crawler page names no webmaster tool, no submission form, and no partner index as a route into ChatGPT's answers. If an article tells you to submit somewhere else to appear here, it is inferring, and it owes you the source.
  • The eligibility statement OpenAI does make is narrow and specific: "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links." That is the only sentence in the crawler documentation connecting a directive on your server to your presence in an answer.

Anything else you read about this engine's retrieval layer is inference until it names a source. Some of it may well be right. None of the pages we opened for this article traces it back to OpenAI, and a plan built on an unsourced claim is a plan built on somebody's guess.

Which OpenAI crawler decides whether you can be cited?

OAI-SearchBot, and OpenAI states it in a single sentence. Four agents are documented, they do four different jobs, and only this one is described as the route into ChatGPT's search answers. The descriptions in the middle column below are OpenAI's own words, quoted from the page linked above.

User agentOpenAI's stated purposeWhat it decides for you
OAI-SearchBot"used to surface websites in search results in ChatGPT's search features"Citation eligibility. OpenAI recommends allowing it and states that sites opted out will not appear in ChatGPT search answers.
GPTBot"used to make our generative AI foundation models more useful and safe"Whether your content can be used in foundation model training. A separate decision with separate consequences.
ChatGPT-User"used for certain user actions in ChatGPT and Custom GPTs"Live, person-initiated fetches. OpenAI states it "is not used for crawling the web in an automatic fashion."
OAI-AdsBot"used to validate the safety of web pages submitted as ads on ChatGPT"Nothing about organic citation. It exists, our own bot table omits it, and it is worth knowing before you meet it in a log.

During our technical audits the common finding here is a robots file that is out of date rather than wrong. Sites that made a decision about OpenAI back when there was one token to decide about are now governing four, and the token that actually gates citation is usually not the one anybody thought about.

The allow-or-block decision is a separate question: our guide to AI crawler access, vendor by vendor sets out what each token buys and what refusing it costs. This section is only about which token the citation depends on.

One timing note, because it changes how you read a test. OpenAI states that "it can take ~24 hours from a site's robots.txt update for our systems to adjust." A log pulled the same afternoon you shipped an edit is not a verdict on the edit.

How is ChatGPT Search different from an answer with no browsing?

They are two different machines sharing one text box, and only one of them can cite you. A search-backed answer retrieves live pages and links them. An answer written without browsing comes from the model, so it can describe your company, get it wrong, and attribute nothing, because there is no source to attribute.

OpenAI's documentation reflects the split in the agents themselves. OAI-SearchBot is the one described as surfacing websites in search results in ChatGPT's search features. ChatGPT-User covers "certain user actions in ChatGPT and Custom GPTs" and, in OpenAI's words, "is not used for crawling the web in an automatic fashion." Those are two genuinely different events arriving at your server.

The consequence people miss: a page can be perfect and still be irrelevant to a given answer, because the browsing decision happens upstream of anything on it. No amount of formatting reaches a response that never left the model.

There is a second, quieter consequence in the same documentation. OpenAI notes of the user-initiated agent that "because these actions are initiated by a user, robots.txt rules may not apply." So one of the two routes to your content is governed by the file you control and the other is documented as possibly not being. Worth knowing before you assume a directive covers everything wearing the same brand name.

Why does ChatGPT cite you in one answer and not the next?

Because three separate gates sit between a question and your link, and you have real input into exactly one of them. Reading them in order is what stops a team from rewriting pages to fix a problem that lives somewhere else entirely.

  1. Did the answer browse at all: if the assistant answered from the model, nobody was cited, and the absence says nothing about your page. Not documented, not configurable, and not something you can observe from outside.
  2. Was your page reachable: either it was in the index that OAI-SearchBot builds, or a person's request triggered a live fetch. This is the documented gate and the only one with a published recommendation attached to it.
  3. Did the writing step use it: a page can be retrieved and still not quoted. OpenAI's crawler documentation says nothing about how sources are chosen or weighted once you are eligible.

Gate two is the one you own. It is also the cheapest to check, which is a rare piece of good news in this category.

Layered on top of all three is ordinary run-to-run variation, which is a measurement problem rather than an engine-specific one. A single run tells you almost nothing on its own, and that reasoning holds across every assistant, so it is not worked through here.

What does ChatGPT Atlas change about how your page gets fetched?

Nothing you can govern through the crawler documentation, and that is the finding rather than a gap in this article. As of September 6, 2026 the word "Atlas" does not appear anywhere on OpenAI's crawler page. The four documented user agents are the complete set of tokens you can write rules against, and a browser sitting on someone's desktop is not among them.

The closest documented analogue is the user-initiated agent, and OpenAI has already told you how that one behaves: these actions are initiated by a person, so robots.txt rules may not apply. Read across from that and the planning assumption becomes reasonable rather than speculative. Traffic generated by a person browsing with an assistant is a person's traffic. It is not a crawler you configure; it is a visitor you should be able to see and serve.

This article does not name the user-agent string Atlas sends. OpenAI's crawler documentation does not say, and a page that does name one is reading its own server logs, which is a different kind of claim. If that answer matters to your architecture, the honest way to get it is from your own logs rather than someone else's.

Treat this section as the one most likely to date: everything in it describes the documentation as it read on September 6, 2026. Re-open the page before you act on it, and re-open it before you quote this article.

What actually earns a ChatGPT citation?

It is worth sorting the documented from the inferred before anyone spends a quarter on the wrong half.

What the crawler documentation covers: that OAI-SearchBot surfaces websites in ChatGPT's search features, that opting it out removes you from search answers while navigational links can still appear, that allowing it in robots.txt and permitting the published IP ranges is the recommendation, and that a robots change takes around a day to register. That is the crawler documentation's complete account of how a page becomes eligible.

What the crawler documentation does not cover: which source gets picked once several are eligible, whether a fresher page displaces an older one, why one domain is quoted where a comparable one is not, or whether anything on the page moves the decision at all. A list of ChatGPT ranking factors is somebody's reverse-engineering unless it names a source, and the page owes you that disclosure.

So there is no ChatGPT writing style to adopt, and if there were, the crawler documentation does not describe it. Once the engine can reach you, the work is the same clarity and corroboration work every retrieval-based surface rewards, which is exactly why this article carries no engine-specific content method.

Both halves of that work are already written down. The readiness checks that apply to every engine deal with whether a page can be parsed and lifted at all, and our pillar on how AI answer engines choose what to cite covers the corroboration that gets a source picked over its neighbours. Neither of them changes because ChatGPT is the engine asking, so neither gets rebuilt here.

Why is ChatGPT eligibility a different problem from AI Overviews?

The two vendors publish different kinds of information, and that difference shapes what you can plan for on each surface. Everything in the Google column below comes from Google's documentation on AI features for site owners and its separate note on generative features in Search. Nothing in the ChatGPT column does.

QuestionGoogle AI OverviewsChatGPT
What does the vendor publish about eligibility?A stated requirement: a page "must be indexed and eligible to be shown in Google Search with a snippet."A stated recommendation to allow OAI-SearchBot, plus the consequence of opting out.
Where is the content retrieved from?Google's own Search index. Its documentation describes generative features as "rooted in our core Search ranking and quality systems."Not stated. The crawler documentation names no index.
Is a special file or markup required?No. Google states that no new machine readable file, "AI text file," markup or Markdown is needed to appear in Search.Not addressed either way in the crawler documentation.
What can you verify yourself?Whether Search Console reports the page as indexed, and whether an Overview actually shows up when you run the query.Your own server logs, split by the four documented user agents.

The last row is the useful one. One vendor hands you a coverage report and no way to see citations; the other hands you nothing centralized and leaves you a complete first-party record in logs you already pay to keep. Neither explains why a particular page was selected, so the honest plan optimizes for eligibility on both surfaces and treats selection as unpublished on both.

How do you check whether ChatGPT is citing you?

Only one of the checks below is a count. The other two are samples, and the order matters, because what the first check returns changes what the other two could possibly mean.

  • The fetch: search your access logs for all four documented user-agent strings, then check the matching requests against the IP ranges OpenAI publishes. A user-agent header is self-reported, so an unverified match proves nothing. This is the only measurement here that you own outright, and it is the one that separates "the engine never arrived" from "the engine arrived and chose something else."
  • The referral: segment analytics sessions arriving from chatgpt.com. Real but partial, because being cited does not require anyone to click. A zero here is not evidence that you are absent from answers.
  • The answer: ask the engine the questions your buyers ask and record what it names and links. This is sampling, and how to sample it properly is engine-agnostic method that lives in our guide to tracking AI search visibility rather than here.

The engine-specific payoff sits in the first check, and it is the reason to bother with logs at all. Because the four agents are distinct strings, your log tells you which route reached you. A ChatGPT-User hit means a person asked and the assistant went and got the page. An OAI-SearchBot hit means the index crawl found you on its own schedule. Those are different situations with different fixes, and a tool that samples answers cannot separate them for you.

Is per-engine ChatGPT work worth it for a B2B company?

Yes for the configuration and the measurement, no for the content, and the gap between those two answers is not close. Everything the crawler documentation covers is about access. None of it is about selection. So a configuration check has a published standard to be checked against, while a ChatGPT-flavored rewrite has nothing at all to aim at.

We have written elsewhere that ChatGPT is the assistant B2B and SaaS buyers reach for first, with Perplexity behind it, and the same set of questions applied to Perplexity reaches a similar verdict for similar reasons. If you are only going to do this exercise deliberately for one engine, that ordering decides which one.

In my experience, the most effective approach is to treat this as a standing configuration check rather than a project. Check that the citation token can reach you, and that what it receives is the actual page rather than a bot challenge or a cut-down variant. Then put the remaining effort into being worth citing on any surface. The second half is where the compounding is, and it pays on every engine at once.

Your situationDoes engine-specific ChatGPT work earn its place?
Your robots.txt predates 2024, or nobody remembers writing the AI rules in itYes, and this is the highest-value hour in the article.
A bot-management layer, WAF or CDN rule set sits in front of your siteYes. When a site is missing from answers and its robots file looks clean, the rule that stopped the fetch was usually written for scrapers and never scoped to AI agents.
Your content is the product you sellYes, though the answer may well be to stay out. Absence is a defensible position here, as long as somebody actually decided on it.
You already run cross-engine AEO work and your logs show clean, verified OAI-SearchBot hitsNo. You are finished here. Spend the time on being corroborated elsewhere instead.

When to skip it entirely: when you have not yet checked the logs. Every recommendation above assumes you know whether the engine is reaching you, and roughly none of them survives finding out that it never has.

The habit worth building here is unfashionable and cheap: when a vendor publishes a crawler page, read the whole page before you read anybody's summary of it. Most of what circulates about this engine is a summary of a summary, and the primary document takes ten minutes.

This week, pull a day of access logs and count how many of the four OpenAI agents actually show up. Then open your robots.txt and search it for the string OAI-SearchBot. If it is not there, read your wildcard rules and work out what they say about it, because silence and a wildcard are not the same answer. Teams that keep AI search inside their content marketing program rather than running it as a side project tend to catch this kind of thing earlier, because somebody is looking.

Need a Content Strategy That Actually Ranks?

We help businesses build topical authority with SEO-driven content that performs in both Google and AI search.

Let's Build Your Content Strategy
FAQ

Frequently Asked Questions

How fast can you rank on ChatGPT?

There is no ranking to reach, so the honest version of the question is how fast you can become eligible to be cited. OpenAI's crawler documentation publishes one interval: it states that a robots.txt change takes around 24 hours for its systems to adjust. Its crawler documentation says nothing about how long a new page takes to be crawled, indexed or selected, so any specific number you see for that was inferred rather than sourced.

How do you rank in ChatGPT answers?

You earn a citation rather than a position. Allow OAI-SearchBot in robots.txt, since OpenAI states that sites opted out of it will not be shown in ChatGPT search answers. Then look at what that agent actually receives from your server, because a bot challenge or an error page blocks the fetch even when the robots file permits it. Past eligibility the crawler documentation says nothing about source selection, so the rest of the job is the clarity and corroboration work every answer engine rewards.

How do you rank number one in ChatGPT results with an AI SEO strategy?

There is no position one. ChatGPT returns a written answer with a small number of linked sources rather than an ordered list, so no first slot exists for anyone to win. The metric that replaces it is coverage: across the questions your buyers actually ask, how often are you named and how often are you linked. That is measured with a fixed set of prompts run repeatedly, not with a rank tracker.

What is SEO for AI called?

Two terms are in common use and they overlap heavily. Answer engine optimization, or AEO, usually refers to making a passage clean and self-contained enough for an engine to lift it. Generative engine optimization, or GEO, usually refers to earning the brand and source signals that get you selected in the first place. No standards body owns either definition, and most real work involves both, so treat the labels as shorthand rather than as distinct disciplines.

Does ChatGPT use Google's index?

OpenAI's crawler documentation does not say. Read on September 6, 2026, it names four user agents, describes what each one is for, and mentions no underlying search index of any kind, whether Google's, Bing's or its own. That is a statement about what a dated document contains, not a statement about the product. Treat any page that names ChatGPT's retrieval layer with confidence as an inference, and check whether it cites a source.

Share This Article

Anshuman Sinha
Written by

Anshuman Sinha

AI SEO Specialist, GrowthHasten

Anshuman Sinha is an AI SEO Specialist and Computer Science Engineer with over three years of experience in SEO and five years in web development. He specializes in Technical SEO, AI Search Optimization (AEO and GEO), SaaS SEO, and building high-performance websites with modern technologies.

View profile

Stay Ahead Of The Curve

Get the latest SEO insights and growth strategies delivered to your inbox. No spam, just actionable advice.