Menu

Cloudflare Sets Deadline to Block AI Crawlers

1 month ago 0

Cloudflare, responsible for managing a significant portion of internet traffic, has set a September deadline for AI crawlers. These crawlers must differentiate between bots gathering content for search and those collecting data for AI training. Failure to comply will result in blockage from pages hosting advertisements.

On March 2025, the Cloudflare office in San Francisco reacted to AI web crawlers’ behavior, emphasizing a need for clear separation of their functions.

By enforcing this deadline, Cloudflare aims to control the impact of AI crawlers on its advertising-hosted pages, ensuring content is gathered appropriately.

Leave a Reply

Leave a Reply

Your email address will not be published. Required fields are marked *