Cloudflare and OpenAI say their joint search pilot is working, so far. In a blog post on October 15, written by Cloudflare's Matthew Conroy and Jack Galilee with OpenAI's Henry Aspegren and James Balfour, the companies shared early results: signals from Cloudflare's network have helped OpenAI find web pages its search crawler did not know about, and those pages are now in the index ChatGPT draws on.
For anyone who runs a website, this is a look at how pages get into ChatGPT Search, and at what the two companies say they will and will not do with site data.
What the pilot is
Cloudflare announced the research pilot on July 8, 2026. The goal, per that release, was to see whether insights from participating websites on Cloudflare's network could help AI search engines discover and index content on the open web more effectively, with a focus on "the accuracy and timeliness of answers." The release named signals "such as content freshness, traffic quality, and actual page changes." Cloudflare says more than 20% of the web sits behind its network.
At launch, Nick Ryder, VP of Research at OpenAI, said: "Up-to-date information is important for delivering accurate answers to people using ChatGPT."
The early result
The October post gives one headline figure. The pilot used "a participating cohort of ~1% of Cloudflare's total network," and it "has enabled OpenAI to discover and index hundreds of millions of new web pages."
The post calls the answer to its research question "a resounding yes." The post does not give numbers for traffic or clicks sent back to those pages, and it does not name the sites in the cohort.
How it works
The division of labor is simple. Cloudflare's network provides signals that help OpenAI's search crawler find pages. OpenAI then "independently evaluates and ranks how those pages appear based on how relevant and helpful they are." In the post's description, ranking stays with OpenAI.
The post argues that useful pages are not always well linked. Its examples are a manual for an exact washing machine model, current passport requirements and fees, and a local restaurant's latest menu and hours. "These pages can be valuable even if few other sites link to them," the authors write.
What they learned about junk URLs
Much of the post is about avoiding wasted crawling. The companies found "how easily automated traffic can make a nonexistent page look interesting." The patterns they list:
- Bots repeatedly ask for files that are not there.
- Some sites redirect almost any address to their homepage.
- Other addresses lead to copies of the same content.
Spotting these patterns lets them cut work "before a crawler fetches the page." Filtering keeps crawlers away from pages that do not exist or that redirect elsewhere. Change signals tell the crawler when a page is worth another visit, and when it is not.
The companies are also testing signals that flag potentially harmful pages. Their example is a legitimate site with one compromised page, such as a fake login form planted by an attacker. The idea is to flag that one page rather than the whole domain. The post says only that "early results suggest" these signals can help identify pages that should be kept out of an index.
What it means for site owners
There is a trade-off, and the post states it. "Finding new pages can initially increase crawling as those pages are evaluated and indexed." The stated goal is to keep the index current while cutting the work it takes, which for Cloudflare customers means "less unnecessary load."
On control, the post makes two commitments:
- "A signal about a page doesn't grant permission to access it."
- The research "uses metadata about the web for search and discovery, rather than sharing page content for model training."
For managing how AI systems reach a site, Cloudflare points owners to its AI Crawl Control tool.
Why it matters
ChatGPT Search links back to relevant sources, which the post says gives people a way to discover "the websites and publishers behind" the content. So a page that is missing from OpenAI's index cannot be cited there. ChatGPT Search has changed how it gathers sources before: in August, Promptwatch data showed a jump in domain-scoped queries. The pilot adds a new way in: network signals, instead of only what a crawler finds on its own, including for pages that few other sites link to.
It also gives Cloudflare a bigger role. The company says more than 20% of the web sits behind its network, and it already offers site owners a tool for managing AI crawlers. Now it is also feeding discovery signals to one AI search engine. The post presents the partnership as good for publishers, but the only result it reports is pages added to OpenAI's index. The authors say outcomes for site owners are next.
What comes next
The next phase, per the post, will "improve the selection of pages, test faster and more reliable change signals, and measure what happens after content is discovered." Cloudflare says it will keep working with OpenAI "to understand how better site discovery can translate into real business outcomes" and will share what it learns.
The post also ties the work to OpenAI's product push, saying it complements GPT-6 and Intelligent UI, which OpenAI is using to make ChatGPT "an even better place to shop, learn, and find answers." For how the GPT-6 models differ, see our guide to which GPT-6 model to use.
