Where do AI citations come from? Get indexed by search engines before you worry about AI crawlers

Cover: web pages reach AI answers as citations through a live search engine lookup, while AI crawler data mostly goes to training
Key Takeaway

Most citation links in AI answers are found through a search engine while the AI is answering, not fetched by AI crawlers. Each Chinese AI platform relies on a different search engine, so brands should first make sure their site is properly indexed by Bing, Sogou, Toutiao, Shenma and Baidu.

Two ways AI uses a web page

In GEO (generative engine optimization) you often hear that AI crawlers will visit your site, so your site should be crawler-friendly. That is true, but only half the story. Most of the clickable citation links in an AI answer were not fetched by an AI crawler. The AI found them through a search engine while it was writing the answer.

There are two very different ways an AI model ends up using a web page:

Training dataWeb search (live retrieval)
Where the data comes fromPublic datasets such as Common Crawl, and training crawlers such as GPTBotA search engine index (the platform's own or a third party's)
What it does for a brandThe model "remembers" the brandThe answer "cites" the page
When it takes effectWhenever the model's training data is next updatedAs soon as the page is indexed, it can be cited
Can a brand influence it?HardlyYes, through indexing and content

Every citation we see when monitoring brand visibility in AI comes from the second path.

Being crawled by AI does not mean being cited

A 2026 University of Toronto study tracked 158,000 AI answers and 468,000 citations over 90 days. The conclusion is clear: every page ChatGPT cited came from a search it ran itself while answering. Whether a page could be found in that search explained 32% of the variation in citations. Page features such as FAQ blocks, structured data and page speed explained only 2.8%.

Training crawlers such as GPTBot and ClaudeBot collect content mainly to train models. They do not produce citations directly. Which page gets cited is decided by retrieval and the search index.

Training data still matters. The same study found that about 46% of the brands ChatGPT mentioned had no cited page behind them: they came from the model's memory. But that is hard to influence in the short term. To raise your chances of being cited, what counts is search engine indexing.

Which search engine sits behind each Chinese AI platform

The main Chinese AI platforms do not use the same search engine for web search, and most of them do not use Baidu:

AI platformMain source for web search
DeepSeekBocha AI search API (per 2025 media reports)
Tencent YuanbaoSogou search and WeChat search (Official Accounts, Channels); a standalone Yuanbao search launched in November 2026
DoubaoToutiao and Douyin in-app search, plus Bing for the wider web
QwenQuark search (Alibaba Cloud web search documentation)
ERNIE Bot, Baidu AI SearchBaidu search

This is drawn mainly from public reports and platform documentation. No platform discloses its full retrieval setup, and it can change between versions. It also explains why DeepSeek and Doubao cite almost entirely different pages: they are not searching the same index.

So a site that is only indexed by Baidu may not be found in DeepSeek, Yuanbao, Doubao or Qwen. Baidu still matters for ERNIE Bot and Baidu AI Search; see our guide to Baidu's AI traffic report.

An observation: a site shut out of Sogou

While monitoring an industrial B2B brand month by month, we noticed that the brand's overseas website returned 403 (access denied) to the crawlers of several Chinese search engines, including Sogou, Toutiao and Quark.

Over the next three months, Yuanbao, which relies on the Sogou and WeChat indexes, cited this site about 40% less. Doubao, Qwen and DeepSeek showed no similar drop.

A reasonable explanation is that the other three platforms can reach the site through other indexes, such as Bing or a third-party search API, while Yuanbao depends mainly on Sogou. One brand is not proof, but it shows that the same website can perform very differently across AI platforms depending on whether each platform's search engine has indexed it. AI sources already change heavily every month; gaps in indexing make it even harder for a brand to stay among them.

An indexing checklist for your website

Based on the above, we suggest checking your website's indexing in this order:

  1. Check the sitemap first. URLs in the sitemap must be complete, live addresses, and robots.txt should state where the sitemap is. Every search engine depends on this, yet it is often overlooked.
  2. Bing: submit the site to Bing Webmaster Tools and turn on IndexNow so new pages are found faster. Bing is linked to both ChatGPT and Doubao.
  3. Sogou and WeChat: submit the site to the Sogou webmaster platform, and republish key content on your WeChat Official Account, one of Yuanbao's most important sources.
  4. Toutiao search: submit the site to the Toutiao search webmaster platform to cover Doubao's ByteDance sources.
  5. Shenma: Alibaba's mobile search engine, in the same family as Quark. Worth submitting at the same time.
  6. Baidu: still important for ERNIE Bot and Baidu AI Search, so keep maintaining it; see our Baidu SEO and SEM service.
  7. Servers and logs: overseas servers and firewall rules can block Chinese crawlers by mistake; see two questions about China websites. Check the access logs for crawlers such as Bingbot, Sogou, Bytespider and YisouSpider regularly, to make sure they are not being refused and are not getting large numbers of 404s.

Checking is simple: in each search engine, search "site:yourdomain" plus a product keyword and see whether your key pages are indexed. For a full view of how your brand performs across Chinese channels, start with a Digital Footprint audit (DFP).

Once indexed, match the way users ask

Indexing decides whether a page can be found. Once it is found, research is consistent about which page gets cited: headings and paragraphs need to match the questions users actually ask.

AI rewrites a user's question into several sub-queries, searches for each, and then picks pages by semantic similarity. A page headed "Product features" will struggle to match a question like "how do I choose this kind of equipment". Writing common questions as headings, with specific figures and answers beneath them, works better than simply adding an FAQ block or structured data. For more on writing for AI, see our practical introduction to GEO.

GEO is not a mysterious new discipline. Citations in AI answers rest on search engine indexing, and each Chinese AI platform sits on a different search engine. The most practical first step is to make sure your site is properly indexed by Bing, Sogou, Toutiao, Shenma and Baidu, then optimize content for the way users ask, and keep monitoring citations on each platform. For how we monitor, see how to diagnose a brand's AI visibility with BrandLens.

For a step-by-step fix for slow Baidu indexing, see Baidu indexes our website slowly; for how DeepSeek picks its citations, see how to get your brand into DeepSeek's answers.

Key Data & Sources

DataValueSource
AI answers / citations tracked (90 days) 158,000 / 468,000 Chen & Koudas, What Drives AI Citations?, 2026
Citation variation explained by whether a page can be found 32% Chen & Koudas, 2026
Citation variation explained by page features 2.8% Chen & Koudas, 2026
Share of ChatGPT brand mentions with no cited page 46% Chen & Koudas, 2026
DeepSeek web search provided by Bocha — National Business Daily, March 2025
Standalone Yuanbao search launched November 2026 Tencent News
Web search supports the Quark search engine — Alibaba Cloud documentation
Anonymous industrial B2B brand: change in Yuanbao citations of its overseas site (3 months) About 40% lower Dékuple China monthly brand AI visibility monitoring, Sept–Nov 2026

FAQ

Should we block AI crawlers such as GPTBot in robots.txt?

Training crawlers such as GPTBot mainly affect model training, not which pages are cited in answers. Whether to block them can depend on copyright and server load. Just make sure you do not block search engine crawlers or the retrieval crawlers of AI platforms, as that directly affects indexing and citations.

Our website already does Baidu SEO. Do we need other indexing?

Yes. Web search in DeepSeek, Yuanbao, Doubao and Qwen relies on indexes such as Bocha, Sogou and WeChat, Toutiao and Bing, and Quark, mostly not Baidu. A site indexed only by Baidu may not be found on these platforms.

How can we tell whether these search engines have indexed our website?

Search "site:yourdomain" plus a product keyword in each search engine and see whether key pages appear. Also check indexing data in each webmaster platform, and the status codes those crawlers receive in your server logs.

Once a page is indexed, will AI definitely cite it?

No. Indexing is only the prerequisite. AI picks the pages that best match the question semantically, so headings and paragraphs should answer users' questions directly with specific information. Monitor monthly to see how citations change on each platform.