Cloudflare has doubled down on its goal of protecting news sites from AI crawlers that scrape and steal their content.
The firm’s Project Galileo, a security plan devised to
protect important civic voices online, now comes with two other Cloudflare programs: bot management and AI crawl control services.
Project Galileo
users—journalists and independent news organizations—can now also protect their websites from AI crawlers. The price: nothing.
Cloudflare debuted Project Galileo in 2014—it
now covers more than 3,000 organizations in 125 countries. The two other entries are of more recent vintage.
Bot management is “a security tool that uses machine learning to
analyze web traffic to distinguish between good bots, like search engine crawlers, and bad bots that attack websites or steal credentials,” the company says in a blog post. “It allows
website owners to block bad bots from reaching their websites, while making sure helpful bots can continue to do their work.”
advertisement
advertisement
AI crawl control provides similar tools: it lets users allow
or block access on a crawler-by-crawler basis.
Why is this important? Because publishers are losing traffic and revenue to AI companies.
“Cloudflare recently published
data that shows with Open AI its 750 times more difficult for website owners to get the same volume of traffic than it was with previous Google search,” the blog post states.
“With Anthropic, it's 30,000 times more difficult.”
Indeed, “The CEO of the Financial Times recently stated that AI had caused a ''pretty sudden and
sustained' decline of 25% to 30% in traffic to its articles arriving via search engines," the post continues.
One advocate of Cloudflare’s approach is LION Publishers, a group
that helps independent news publishers build more sustainable businesses.
"Independent publishers need tools that are easy to use and affordable, so they can focus on growing their
business,” says Sarah Gustavus Lim, membership director of LION Publishers. “LION appreciates the security and protection Cloudflare has provided our members through Project
Galileo for years, and we're excited to see more resources now available to help members manage the rapidly evolving landscape of digital security." -