While I get your point about Anubis excluding some users, its purpose is to protect the "commons" from those who would abuse and destroy them: big tech crawlers that do not respect robots.txt files.