If you do some research and cold-call their previous employers/clients, you'll usually find out if the relationship was a negative experience for them. If you get a couple of these, and they've come to that conclusion independently, then you have your answer.
that solves the first problem (the applicant only choosing to give you the contact info of people who are likely to give them positive marks) but it doesn't help with the second issue.
I don't see any case where it's in the best interest of anyone to say negative things about a past employee. And I don't think you can count all "company policy is that we simply confirm they worked here" messages as negative, because sometimes that really is company policy.
When asked for a reference, have you ever said anything negative about a past employee or co-worker? Have you ever heard anything negative about someone you were considering hiring after cold calling a previous employer?
I haven't, but I know quite a few people who have checked references and gotten a negative response - in one case derisive laughter followed by an incredulous "Really? They put me as a reference?". Obviously it depends on whether it's a big shop where you'll get HR stonewalled, or a smaller place where you'll have the owner or a coworker.