The best reason is because downvotes piss people off. They have been the cause of more flamewars on reddit than anything else. When you know you are right but everyone is downvoting you and upvoting the "wrong" guy, I know very few people who won't react aggressively.
This is why democracy + free speech (in theory, at least) works. No matter how unpopular, a voice cannot be silenced. If you don't like a person's viewpoints, you can't shut them down by downvoting them; you can only upvote the viewpoint you do like, and / or present your own position yourself, if no one else has done it.
