In a debate likely to get louder as AI agent usage grows, some people say Perplexity crawling blocked websites isn't a simple matter.

Cloudflare is introducing a way to charge AI web scrapers Content creators can protect their sites from unwanted scrapers Specific crawlers can be granted free access, charged, or blocked Online ...

Internet giant Cloudflare says it detected Perplexity crawling and scraping websites, even after customers had added technical blocks telling Perplexity not to scrape their pages.

Cloudflare pushes Google to separate bots for AI Overviews and search indexing.

Cloudflare once again comes under pressure for enabling abusive sites Cloudflare masks the origin of roughly 10% of abusive domains, watchdog says.

Cloudflare reshapes the AI landscape by blocking Big Tech AI bot crawlers by default, allowing websites to demand payment for content access.