> The latest was a slow loris approach where it takes forever for robots.txt to download

Applying penalties that exclusively hurt people who are trying to be respectful seems counterproductive.