As of September 15, Cloudflare's Block and Block on pages with ads settings apply to mixed-use crawlers, a category that includes Applebot, Bingbot, and Googlebot. These are the same bots that index pages for search results and also feed data to AI training pipelines. Cloudflare's announcement frames the change as separating two outcomes that used to be tangled together: staying visible in search while refusing to hand over content for model training. The new Disallow AI Training option is built for that split. Sites that previously relied on a blanket Block rule to keep AI crawlers out may now be blocking the crawlers that put them in search results at all.
The timing matters because Cloudflare is retiring the tools many operators already configured. Block AI Bots and Managed Robots.txt are being deprecated, replaced by a set of Search, Training, and Agent controls plus something called Bot Preference Sync. Anyone who set a rule months ago and never touched it again is now running on defaults that behave differently than when they were set. Cloudflare's post does not say how many zones were migrated automatically versus how many require manual reconfiguration, and it does not disclose a rollback window or an opt-out path for teams that want the old behavior preserved during a transition period.
For on-call engineers, the risk is not an outage in the usual sense. It is a slow bleed: search ranking drops, referral traffic declines, or a client reports a page missing from results, and the root cause traces back to a bot rule nobody reviewed after the September 15 change. Check whether any zone uses Block or Block on pages with ads at the account or ruleset level, and confirm whether Applebot, Bingbot, or Googlebot appear in the blocked traffic logs. If they do, the fix is moving training-specific exclusions to Disallow AI Training rather than relying on a single block rule to do both jobs.
Mixed-use crawlers are the mechanism that makes this consequential. A single bot fetching a page for search indexing and AI training under one user agent means a rule written to stop one effect now stops both, unless the platform draws a finer line. Cloudflare's redesign tries to draw that line at the control level instead of the bot level. That shifts the burden onto operators to re-declare intent explicitly rather than infer it from a single toggle. Nothing in the announcement specifies a deadline for full deprecation of Block AI Bots, so teams running on the old control have no stated clock, only the knowledge that it is going away.
Bot Preference Sync is the piece worth watching next, since Cloudflare has not detailed how conflicts between search-engine indexing preferences and training-block preferences get resolved when they disagree on the same crawler. Until that is documented, assume any crawler-blocking rule written before September 15 needs a manual audit, not an automatic pass.