Cloudflare announced Bot Preference Sync, a tool available across all plan tiers from Free to Enterprise that automatically generates or updates a site's robots.txt to match its AI bot configuration, keeping stated preferences and edge enforcement in sync1. The feature supports separate controls for Search, Agent, and Training AI traffic, with options to Allow, Block on ad-serving pages, or Block everywhere for Search and Agent categories. Cloudflare said Bot Preference Sync will be on by default for all new customers and will roll out to all existing customers in the coming week.
Cloudflare Launches Bot Preference Sync to Unify AI Bot Controls
Cloudflare announced Bot Preference Sync, a free-tier tool that auto-syncs robots.txt with AI bot settings for Search, Agent, and Training traffic.