Perplexity is an answer engine that runs its own crawler and its own index rather than reading Google's, and that one fact decides what you can control here and what you can only sample. Anyone running answer-engine work across several surfaces eventually hits the question of whether this one needs its own budget line. It does, but not the line most teams expect: what follows covers Perplexity's two documented user agents, the access decision they create, how a citation here differs from a link in a Google AI Overview, what the publisher program means for a company that is not a publisher, and how to tell whether any of it is working. It does not repeat the on-page method, which is engine-agnostic and lives in the six checks that decide whether a passage can be extracted. Where something is documented I say so, and where the evidence stops I say that too.
The short version
- Most Perplexity problems are access problems, not writing problems. Three separate layers can stop the engine at your door, and only one of them is
robots.txt.PerplexityBotandPerplexity-Userare two different things. Perplexity's documentation states that the second one generally ignoresrobots.txt, because a person asked for that fetch.- Perplexity has published nothing about how it selects or ranks the sources it cites. Any page listing its ranking factors is inferring, and owes you that disclosure.
- Allowing the crawler is not the same as being reachable by it. Perplexity asks you to permit its published IP ranges as well, which is a firewall decision rather than a robots decision.
- For most B2B teams, Perplexity needs no separate content work. It needs separate access and separate measurement, and that is an afternoon rather than a quarter.
Does Perplexity use Google's index?
No, and Perplexity's own documentation is the evidence rather than an inference. Perplexity's crawler documentation says the company collects data using web crawlers and user agents "that gather and index information from the internet," and it describes PerplexityBot as "designed to surface and link websites in search results on Perplexity." A company that runs its own crawl and its own index is not querying somebody else's.
Three of our own guides state this in passing and none of them explains it. In our answer engine optimization guide, our GEO guide, and our comparison of the two, the sentence does one job: it stops a Google statement from being read as a statement about the whole category. That is a correct use of the fact and it is not an explanation. So here is the explanation.
What follows from it:
- Two access decisions, not one: being crawled and indexed by Google says nothing about whether Perplexity can reach you. Two crawls, two sets of directives, no shared status anywhere. A site can be fully covered in one and invisible in the other, with nothing in either interface to say so.
- Googlebot rules do not travel: a
User-agent: Googlebotblock, aGoogle-Extendedopt-out, or anosnippettag aimed at Search does not governPerplexityBot. Perplexity needs its own named block, in the same file. - Google's AI-features guidance is out of scope: the eligibility rules Google publishes for AI Overviews and AI Mode are about Google's index. Read as category-wide guidance they become a scope error, and it is the one this site keeps correcting.
What does not follow from it: that Perplexity rewards different writing. Both surfaces retrieve passages and have a model write over them, so the clarity work transfers. The difference sits upstream of the copy, in who may fetch the page and whose index it lands in.
One boundary, stated plainly. "Not built on Google's index" is a claim about crawling and access. It is not a claim about ranking logic, because Perplexity has published none.
What is PerplexityBot, and should you allow it?
Allow it, unless the content itself is the product you sell. Perplexity's documentation is explicit about it: "we recommend allowing PerplexityBot in your site's robots.txt file and permitting requests from our published IP ranges" that it lists on the same page.
Notice how short that page's list is. It documents exactly two user agents, and says of each that it is not used to gather content for AI foundation models. Our guide to which AI crawlers to allow and which to block covers the wider landscape of AI crawlers and the general allow-or-block decision. What follows is the Perplexity row of it, opened up.
| Agent | What Perplexity documents it does | What disallowing it costs you |
|---|---|---|
PerplexityBot | Crawls and indexes pages so they can be surfaced and linked in Perplexity's search results. Perplexity states it is not used to crawl content for AI foundation models. | Your exit from Perplexity's index. This is the one directive that decides whether the engine can cite you at all. |
Perplexity-User | Visits a page when a person's question needs it, and links that page in the response. Perplexity states it is not used for web crawling or for training. | Very little, by this route. Perplexity documents that "since a user requested the fetch, this fetcher generally ignores robots.txt rules." Stopping it takes a firewall, not a directive. |
The directives themselves are ordinary:
- Allow the indexer:
User-agent: PerplexityBotfollowed byAllow: /, or simply no disallow rule matching it. - Refuse the indexer:
User-agent: PerplexityBotfollowed byDisallow: /. Expect to be dropped from the engine's answers over time, not instantly. - Give yourself a day: Perplexity's documentation warns that "it may take up to 24 hours for our systems to reflect changes." Do not read a same-day log as a verdict on an edit you shipped that morning.
In our implementation work the failure we hit most often is not a deliberate block. It is a wildcard rule written years ago for scrapers, or a tidy-up that turned a permissive file into an allowlist and quietly excluded every agent nobody thought to name. Both look intentional in the file and neither was anyone's decision.
Can robots.txt alone control Perplexity's access?
No. Three layers decide whether the engine reaches your page, they fail independently, and a clean robots.txt tells you about one of them. Check them in this order, because each one makes the next check meaningful.
- Layer 1, the directive layer: load your
robots.txtand look for the stringPerplexityBot. Then read the wildcard blocks, not only the named ones, because aUser-agent: *disallow governs every agent you never mentioned. This layer is free to check and takes a minute. - Layer 2, the network layer: Perplexity publishes WAF guidance for Cloudflare and AWS WAF, plus IP endpoints for both agents. It exists because a firewall in front of your site can refuse a bot that
robots.txtwelcomes, and neither system knows about the other. Perplexity's own recommendation is to match on the user-agent string and the source IP, which is also how to read your logs: a user-agent is a header anyone can set. - Layer 3, the response layer: confirm what your stack actually returns to those agents. A bot-challenge page, a 403 from a rate limiter, or a stripped-down variant served to non-browser clients all read as healthy in an uptime dashboard and as a failure to the engine. Server behavior, not page formatting, and the layer teams almost never inspect.
Layer 2 holds the interesting failures precisely because a different team owns it, and layer 3 is invisible without a log. Start from the copy instead and you can spend a quarter rewriting pages the engine never successfully fetched.
Is Your Site Readable By Answer Engines?
Run your site through GrowthHasten's free Website SEO Audit. It crawls the site, scores AI search readiness alongside technical SEO, and names the pages holding you back.
Run the Website SEO AuditWhat actually earns a Perplexity citation?
Nobody outside Perplexity knows, and the useful version of this section labels its own evidence instead of papering over the gap. Perplexity has published no ranking factors, so any list of them was inferred from behavior by whoever wrote it and should say so. Here is the same territory, sorted by what each claim rests on.
Documented by Perplexity: that it crawls and indexes the web itself, the two user agents and what each is for, the recommendation to allow the indexer and its IP ranges, the roughly 24-hour propagation window, the WAF guidance, and that answers have carried citations "from day one," per its publisher announcement. That is the whole of what Perplexity publishes about how a page reaches an answer.
Not documented by Perplexity: how sources are selected, how they are weighted, how recency is weighted in organic source selection, or what makes one domain more citable than another. Our guide to AI search as a whole records that the major engine operators have not released citation formulas for their assistants. Perplexity is no exception.
Documented, engine-specific, and the part that matters: Perplexity documents two routes to your page. There is the indexed route, where PerplexityBot crawls you in advance, and the live route, where Perplexity-User fetches a page because a person's question needed it. Google documents a single route into its AI features, and it runs through Search eligibility: no index entry, no snippet permission, no candidacy. That is the sharpest documented difference between the two surfaces, and it is an access difference rather than a writing difference.
Inferred, and labeled as such: that passage-level clarity earns citations here for the same reason it earns them elsewhere, because retrieve-then-generate is the shared architecture. That is a reasonable read of the mechanism, not a Perplexity statement. It is also why this article carries no Perplexity-specific content method: the extraction work is engine-agnostic, we have published it already, and restating it under a brand name would add a URL and no knowledge.
How is this different from ranking in Google AI Overviews?
The two surfaces differ at the access and verification layer, not at the writing layer. Everything in the middle column below comes from Google's documentation for site owners on its AI features, and nothing in the right-hand column does. Keeping those sources apart is the discipline the whole comparison rests on.
| Question | Google AI Overviews | Perplexity |
|---|---|---|
| Whose index decides eligibility? | Google's. Its documentation requires a page to be indexed and snippet-eligible in Search, and states there are "no additional technical requirements." | Perplexity's own. PerplexityBot crawls and indexes for it, and Google indexing is not part of the stated requirement. |
| What controls access? | Search's own crawl and snippet controls, plus the Google-Extended token for training and grounding in some of Google's other systems. | PerplexityBot in robots.txt, plus your firewall and Perplexity's published IP ranges. |
| Is a strong ranking a prerequisite? | In practice usually, since retrieval draws on pages already eligible in Search. How a Google-grounded surface picks its sources covers that in full. | Unknown. Perplexity has published no eligibility or ranking statement beyond crawl access. |
| Can a live fetch reach a page the index missed? | Not documented as a path. | Yes. Perplexity-User fetches on a person's request, and Perplexity documents that it generally ignores robots.txt. |
| What can you verify yourself? | Coverage in Search Console, and whether an Overview appears for your query. | Your own server logs for both agents, and whether an answer cites you. |
The last row carries the most. Google gives you a coverage tool and no citation report; Perplexity gives you the reverse, a first-party record of every fetch in logs you already pay to store. On each surface, the measurement you can trust most is the one the other cannot supply.
Does Perplexity's publisher program matter if you are not a publisher?
Barely, though the reason is instructive. Perplexity announced its Publishers' Program on July 30, 2024 with six launch partners, including TIME, Fortune and The Texas Tribune. The mechanism was a revenue share: when Perplexity earns revenue from an interaction referencing a partner's content, that partner earns a share, alongside free API access and enterprise seats. In August 2025 the company extended the idea into Comet Plus, a subscription that it says allocates revenue to participating publishers across three kinds of traffic: human visits, search citations and agent actions.
For a SaaS or B2B technology company the practical answer is short: this is a partner program for media organizations and creators, and joining it is not a visibility lever. Note which way the mechanism runs, though. Being cited is what triggers a payment, so the program rewards citation rather than producing it.
Note instead that a vendor's own compensation model now treats search citations and agent actions as three separate categories. That is commercial framing rather than market evidence, and it sits oddly beside your own reporting, where the three almost certainly collapse into one number.
How do you measure whether Perplexity is citing you?
Three different questions. Teams answer them as one, and that is where the measurement goes wrong on this surface. Take them in this order, because they get progressively less certain and progressively more expensive.
Question 1. Did the engine reach you: filter your server or CDN logs for the two documented user-agent strings, then verify the matching requests against Perplexity's published IP endpoints. A first-party fact you own outright, free beyond the query, and the only part of this measurement that is not a sample. Skip the verification and you are counting anything that chose to send that header.
Question 2. Did an answer cite you: this is sampling, not counting, and the accuracy ceiling is set by the engine rather than by the tool you buy. Why two runs of the same prompt disagree covers non-determinism, personalization and run counts, and all of it applies here unchanged. The Perplexity-specific note is smaller: attributions sit inline in every answer, so a manual spot check is unusually readable, which makes it tempting to treat five prompts as a measurement. Five prompts is an anecdote.
Question 3. Did anyone arrive: segment referrals from Perplexity's domain in your analytics, and treat a low or zero figure as inconclusive rather than as proof you are uncited. A citation does not require a click, which is the premise of the whole surface. Most teams start here because it is already on a dashboard, conclude nothing, and never reach question 1.
Is optimizing for Perplexity worth the effort for a B2B company?
Not as a separate content workstream. Yes as a separate access and measurement check, which is an afternoon rather than a quarter. It splits that way for one reason: the access target is documented and the content target is not. A directive, an IP range and a log line can all be verified. A ranking factor nobody has published cannot, so per-engine content work bills real hours against an unpublished specification.
From my experience the ratio is lopsided in a way that surprises people: on this surface, access problems outnumber content problems, and they usually belong to whoever configured the CDN rather than to whoever writes the pages. That is an uncomfortable finding for a content team, because the fix is a ticket in someone else's backlog.
Four conditions that change the answer:
- Your buyers actually use it: for most B2B and SaaS audiences the primary assistant is still ChatGPT, with Perplexity second. Let your own category's answers set the priority rather than the news cycle.
- You sit behind a bot-management layer: then the access check is not optional, because the most likely reason you are absent is a rule you did not write. This is the one case where the work is urgent rather than merely tidy.
- Your content is the product: paywalled research, proprietary datasets and original journalism are the real cases for refusing the crawl. Choosing to be absent is a legitimate answer to this article. It should be a choice rather than an accident.
- You publish data other people cite: original numbers get referenced across engines, and that work pays on every surface at once, which is exactly why it beats a per-engine push.
When to skip it entirely: when the logs show no fetch at all, because no amount of formatting reaches an engine that never completed a request. The general cases for not doing this work, which apply to every engine rather than this one, are in our guide to when answer engine optimization is not worth doing.
One habit is worth more here than any strategy: read your own logs before you read anybody's dashboard. Logs are first-party, complete and dull, and on this surface they are the only place a question gets a definite answer.
This week, filter a week of server logs for PerplexityBot and Perplexity-User, then open your robots.txt and confirm the first one is allowed. If the logs are empty and the file is clean, your next conversation is with whoever owns the firewall, not with whoever owns the copy. If you would rather have content and AI search optimization run as one system, that is the work we do.
Need a Content Strategy That Actually Ranks?
We help businesses build topical authority with SEO-driven content that performs in both Google and AI search.
Let's Build Your Content StrategyFrequently Asked Questions
What is Perplexity in SEO?
Perplexity is an answer engine that crawls and indexes the web with its own crawler, PerplexityBot, then writes answers that cite the pages it used. In SEO terms it is a separate retrieval surface from Google: being indexed by Google does not make you reachable by Perplexity, and Perplexity needs its own line in robots.txt. Success here is measured in citations inside answers rather than positions in a list.
Is Perplexity good for SEO?
As a research tool, yes for some tasks. It summarizes sources and links them, which makes it useful for scanning a topic or checking a claim quickly. That is a different question from whether it sends you traffic. If you mean whether being cited by Perplexity is worth pursuing, the honest answer for most B2B teams is that the access and measurement work pays and separate per-engine content work usually does not.
What is the Perplexity controversy?
It is a dispute over whether AI answer engines should compensate publishers whose journalism they use. In October 2024, Dow Jones and the New York Post filed a copyright suit against Perplexity, which publicly disputed the characterization. Perplexity's own responses include its Publishers' Program and, later, Comet Plus, both of which share revenue with participating publishers. For a site owner the practical takeaway is narrower: decide crawler access deliberately rather than inheriting it.
Is optimizing for Perplexity worth the effort yet?
For most B2B teams, not as a separate content project, and yes as a separate access check. The reason is asymmetric evidence. Perplexity documents exactly how to let its crawler in, and documents nothing about how it selects sources. Verifying robots.txt, firewall rules and server logs takes an afternoon against a documented target. Rewriting pages for one engine spends weeks against an unpublished one.
How do you optimize content for Perplexity without guessing?
By separating what is documented from what is inferred, and acting only on the first. Documented: which user agents exist, that PerplexityBot should be allowed in robots.txt, that its published IP ranges should be permitted, and that changes can take about a day to propagate. Not documented: anything about selection or ranking. So fix access with confidence, apply engine-agnostic extraction work to the page, and treat any ranking-factor list as inference.
How do I get cited by Perplexity?
Start by confirming the engine can reach you. Allow PerplexityBot in robots.txt, check that your firewall or CDN is not blocking it, and confirm your server returns the real page rather than a challenge or an error to that user agent. After that the work is the same extraction and authority work every answer engine rewards, because Perplexity has published no engine-specific requirements beyond access.

Anshuman Sinha
AI SEO Specialist, GrowthHasten
Anshuman Sinha is an AI SEO Specialist and Computer Science Engineer with over three years of experience in SEO and five years in web development. He specializes in Technical SEO, AI Search Optimization (AEO and GEO), SaaS SEO, and building high-performance websites with modern technologies.
View profile



