This is a different question from "is my site being crawled," and it trips people up because fixing crawl access doesn't automatically fix citation. You can be fully crawlable and still never get cited.
Two separate Perplexity crawlers
- PerplexityBot: general crawling that builds Perplexity's index ahead of time. It respects robots.txt; a disallow rule means Perplexity won't index that content at all.
- Perplexity-User: a live fetch triggered by an actual user's question, used to pull a real-time answer.
If either is blocked in robots.txt (directly, or via a wildcard AI-bot rule), that's an easy, mechanical fix. But most sites asking this question have already ruled that out; robots.txt is open, and they're still not getting cited. That's a retrieval and trust problem, not a crawl-access problem.
Why crawlable pages still don't get picked
- No clean, extractable answer on the page: the actual fact or figure is spread across a page, implied by a chart, or wrapped in marketing language instead of stated plainly.
- Weak topical signal: the content doesn't clearly match the question being asked, even if it's tangentially related.
- No structured data: Perplexity, like other retrieval systems, leans on structured data and clear headings to confirm what a page is actually about.
- Thin authority: retrieval and citation systems weight pages that look independently corroborated (mentioned elsewhere, linked to, consistent across the web) over an isolated, unlinked page making the same claim.
- Heavy client-side rendering: if the content that answers the question only exists after JavaScript runs, a crawl-time fetch may never see it.
What actually moves the needle
- State the specific fact, number, or answer plainly, near the top of the relevant section. Don't just imply it via surrounding context.
- Use real heading structure so the page's topic and sub-topics are unambiguous to a system parsing it in one pass.
- Add structured data (FAQPage, Article, Product) that matches what's actually on the page.
- Server-render the content that matters, rather than relying on client-side JavaScript to fill it in.
Fetchling runs live citation tests: it actually asks Perplexity and Claude real buyer-intent questions and checks whether your site gets cited, instead of guessing from crawlability alone.
← all posts