Simon Willison’s Weblog

Your site, your rules: new AI traffic options for all customers (via) In which Cloudflare effectively declare war on Google, Microsoft, and Apple, halfway through a blog post about their new AI traffic blocking tools - emphasis mine:

Another change that will apply on September 15 is that multi-purpose crawlers (specifically those that combine Search with Training) will be allowed/blocked according to all of their behaviors, in line with our call for transparency for website owners. Since the defaults will be enforced by the most restrictive applicable rules, multi-purpose crawlers such as Googlebot, Applebot, and BingBot will be blocked by customers who have selected to block Training (either through the new options to manage AI traffic, or through the legacy Block AI bots service).

The fact that Google (and apparently Microsoft as well) use the same robots.txt user agent for creating their search index and for training AI models has long struck me as deeply unfair: it means you can't opt out of training without opting out of search, while also putting other model training efforts at a competitive disadvantage.

I wonder if Cloudflare's influence is heavy enough to force them to reconsider this policy.