Example of AI not honoring robots.txt
The AI scraper (I can only assume thats what they are) scourge continued, and intensified in the last week. This time they were hitting pagure.io really quite hard.
Robots.txt is only honor system. One of many postings calling it out.
Quote Citation: Kevin, “Mid March infra bits 2025”, 2025-03-15 17:52, https://www.scrye.com/blogs/nirik/posts/2025/03/15/mid-march-infra-bits-2025/