Patreon, a membership platform for creators, is cracking down on AI scraping content for training purposes. The company announced Thursday that it is working with internet infrastructure provider Cloudflare to directly block unauthorized access to AI bots designed to train its AI models based on creators’ work.
The company said the enhanced measures were necessary because AI scraping has become more sophisticated since it first introduced measures to thwart AI crawlers in 2023. Additionally, Patreon’s paywall has long locked much of its creators’ content out of the reach of crawlers. But just recently, the company introduced new discovery tools like a redesigned home feed and Tweet-like Quips, which could expose more content to crawlers.
This shift comes as more online publishers and content creators understand how AI is feeding into their work to make their AI models smarter. To combat this, Cloudflare now offers tools that allow website publishers to limit AI bots. This includes a marketplace called Pay Per Crawl that allows websites to charge AI bots for scraping. Earlier this month, the company changed its policy so that “mixed-use” crawlers — crawlers that both index and train a website’s content — are now blocked by default on all pages hosting ads.
Patreon said it will expand on its existing work with Cloudflare and use the company’s AI crawl control technology to update its AI policies and enforcement tools. The difference here is that rather than simply requiring AI crawlers not to scrape content using robots.txt files (the standard method of providing bots with instructions on how to use the site), Patreon actively blocks AI training bots.
“Consent should not depend on whether the scraper takes action,” the Patreon blog post explains, pointing to stricter measures.
When testing this feature, the post says that weekly attempts by individual AI training crawlers to access Patreon went from “thousands to zero.” This indicates that the AI scraper was ignoring Patreon’s robots.txt file and scraping the site despite our requests.
However, the company said it will allow bots to index pages and organize information that can be used to send users back to Patreon.
“As AI agents become increasingly powerful and popular, creators have the right to have a meaningful say in how their work is used by AI companies,” Patreon product chief Drew Lowney said in the announcement. “For most of the internet, creators have to accept AI training on their work just to reach and grow their audience. Patreon has a different vision: Creators should be able to grow their audience and control how their work is used.”
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
