Similar to This
Google search yes, model training no: New Cloudflare rules
Google search yes, model training no: New Cloudflare rules: Block AI crawlers, but keep them in search results: Cloudflare defuses the conflict with ...
heise.de/news/Google-Suche-ja-Modelltraining-nein-Neue-Cloudflare-Regeln-11457767.html
OpenStreetMap is concerned: Thousands of AI bots are collecting data
OpenStreetMap is concerned: Thousands of AI bots are collecting data: Bots are said to be harvesting OpenStreetMap data on a large scale. This costs money and ...
heise.de/news/OpenStreetMaps-sorgt-sich-Tausende-KI-Bots-erfassen-Daten-11156876.html
Internet: Cloudflare measures 19 percent more traffic in 2025 - | Golem.de
Internet: Cloudflare measures 19 percent more traffic in 2025 - | Golem.de: Google remains the most popular search engine. But AI crawlers now also account for a ...
golem.de/news/internet-cloudflare-misst-19-prozent-mehr-traffic-im-jahr-2025-2512-203404.html
Cloudflare goes after Google AI Overviews with new license for the web
Cloudflare goes after Google AI Overviews with new license for the web: Cloudflare is enhancing robots.txt, giving website owners more control over how AI ...
businessinsider.com/cloudflare-google-ai-overviews-license-bots-scraping-content-2025-9
Death of the Internet: How Google and Chatbots Destroy Websites
Death of the Internet: How Google and Chatbots Destroy Websites: People once surfed the web; today, digital intruders siphon off knowledge from websites ...
nzz.ch/technologie/tod-des-internets-wie-google-und-chatbots-webseiten-zestoeren-ld.1898893
Mediawatch: AI and a last ditch bid for media to claw back online revenue
Mediawatch: AI and a last ditch bid for media to claw back online revenue: The first online revolution undermined the media's business model based on ads ...
rnz.co.nz/news/mediawatch/570889/mediawatch-ai-and-a-last-ditch-bid-for-media-to-claw-back-online-revenue
Data theft allegation: How Perplexity allegedly circumvents website blocks ...
Data theft allegation: How Perplexity allegedly circumvents website blocks for AI bots
t3n.de/news/perplexity-schatten-crawler-1700788/
Cloudflare wants Google to let sites block AI without losing search ...
Cloudflare wants Google to let sites block AI without losing search visibility: Cloudflare recently announced a new pay-per-crawl system aimed at pushing back ...
techspot.com/news/108627-cloudflare-wants-google-sites-block-ai-without-losing.html
Millions of websites to get 'game-changing' AI bot blocker
Millions of websites to get 'game-changing' AI bot blocker: Publishers including Condé Nast and Sky News have welcomed the new tech from internet infrastructure ...
bbc.com/news/articles/cvg885p923jo
Key FactsCloudflare Defaults to Blocking AI Crawlers July 1
Jul 2, 2025 · Reading time: ~45 sec.
Summary created with AI assistance · Source: zdnet.com
  • From July 1, Cloudflare automatically blocks AI web crawlers that access sites without explicit authorization to prevent site slowdowns.

  • Website owners have experienced overwhelming request volumes from AI bots like GPTBot, which often ignore robots.txt rules and cause traffic issues.

  • Cloudflare's new 'Pay Per Crawl' initiative allows publishers to charge AI firms for legal access to their content.

  • This policy aims to protect publishers' traffic and create a fair economic model for creators and AI companies alike.
  • Its new policy is meant to 'give publishers the control they deserve and build a new economic model that works for everyone—creators, consumers, tomorrow's AI founders, and the future of the web itself.'
    — Matthew Prince, CEO of Cloudflare, discussing the company's new policy to block AI crawlers.
The Download: tripping with AI, and blocking crawler bots
The Download: tripping with AI, and blocking crawler bots: Plus: the US Senate has killed an AI regulation proposal
technologyreview.com/2025/07/01/1119520/the-download-tripping-with-ai-and-blocking-crawler-bots/
Cloudflare CEO warns AI crawlers and summaries are eroding the ...
Cloudflare CEO warns AI crawlers and summaries are eroding the internet's business model
techspot.com/news/108399-cloudflare-ceo-warns-ai-crawlers-summaries-eroding-internet.html