I've found a good way to actually figure out how many people are coming is to require people to actually write a message to say they're going. If you just hit the "going" button and don't reply saying you're actually coming, I assume you have no intention of showing up.

Also, it's incredibly rude to show up to an event 4 hours late. 30 minutes, I can understand, if you got caught up, but 4 hours is taking the piss.