The University of East Anglia have a live blog as well that updates their predictions based on statistical model [0] by Dr Chris Hanretty. Predictions are based on how results differ from the prior model. The model is explained in more detail here [1].
Interesting because the odds react far quicker than any live blog I've found, so a sudden swing means that there's just been a surprising result announced.
I don't care about the result. I'm more interested in how our statistical modelling tools failed us. Thoughts? Did opinions change overnight? Did the polling miss a demographic?
Lots of people say in public that they're in favor of immigration, unity, and tolerance, because lots of people will call you a bigot if you don't, but in private, harbor old-fashioned ideas about nationalism, and well, polling involves a human and voting happens in private.
Well, since we have much more information about presidential elections (which happen regularly) than we have about brexit (which is mostly without precedent), you could argue that election 2016 forecasts are probably more accurate.
I don't think that's the case though. I predict a Trump victory on the basis of a very strong social desirability bias and an immense amount of silent resentment against current social justice orthodoxy, especially as it pertains to speech restrictions on campus and in the workplace.
It looks like the average was around 35%:
http://predictwise.com/politics/uk-politics
Interestingly the odds went down pretty sharply as the election came closer, but it is not really fair to pick the lowest odds when evaluating how good the predictions were. But even 15% odds are not really all that unlikely, about a 1 in 6 chance. It isn't as though pundits were predicting leave.
I think betting markets are prone to wishful thinking; there's no guarantee that errors of reasoning toward one side or the other will cancel out, since it's perfectly reasonable to imagine that errors in one direction are more common.
There's no guarantee, but the bookie is watching his, uh, book and adjusting the odds to make sure the payouts come from losing bets rather than his pocket.
I think that hedging would have the opposite effect. The pound is tanking (30-year low right now), so if your salary is paid in pounds, it would have made sense to bet on Leave. I'm not sure who wins economically from leaving. British companies that compete with continental ones?
Pollsters were never confident of this result, with their adjusted and unadjusted results being all over the place, and were cautiously predicting Leave victories as recently as last week. There weren't any relevant precedents for them to test their models against, and the biggest factor in the Leave vote appears to be high turnout from demographics that rarely vote in other elections.
I've found it odd that the markets were far more confident in poll findings than the pollsters but assume that the bets were mainly placed on the simple heuristic that poll results tend to understate voters caution and attachment to the status quo.
Given that there will be a number of swings maybe a good strategy is to bet on the one with longer odds, wait till the next swing, sell and buy the other, rinse and repeat.
Why does the drop from p=0.32 to p=0.03 seem crazy/bogus? Isn't that what you'd expect as the probability distribution both narrows (with more evidence) and moves toward the 'leave' side?
That Remain pulled ahead is somewhat irrelevant though. That seems to be mostly due to London reporting in, which was always expected to vote Remain. What's relevant (for this model at least) is whether the results are higher or lower than predicted per area.
If the model is being updated based on results as they come in, and the results coming in are not randomly distributed, then the updates will be of questionable value. In particular, this update came when a large number of predicted pro-leave results had come in, and no results from predicted strong pro-remain results had come in, so I'm not sure it has much value as a prediction.
Interesting he's since increased the certainty of a Leave vote even after a couple of unexpectedly strong pro-Remain votes swung the betting markets back in favour of Remain
Whether that's because he's better than the markets at modelling differential turnout or the markets know things his confirmed results data doesn't about predicted results in places like Birmingham remains to be seen...
As I understand it, the model is based on the difference between expected and actual results in each area. So the order that results come in should not affect the prediction.
> So the order that results come in should not affect the prediction.
This model uses a frequentist prediction interval, which assumes independently drawn samples, meaning reporting order must be random for the assumptions to be valid. If reporting is non-random, e.g. how early or late a district reports is correlated with things like region, demographics, population density, etc., then the prediction interval is probably narrower than it should be, especially early on in the reporting (meaning the model is overconfident in its prediction).
The headline prediction is more robust if you just want to know which outcome is more likely given current results, but the probabilities being badly calibrated due to these kinds of model assumptions is a common issue in quantitative polisci models.
The totals will go back and forth, but it's all about turnout proportionally in in- vs out- regions versus original projections. Glasgow was expected to be massively pro-remain but did Glasgow turn out in higher/lower numbers than anticipated? and did Glasgow go more or less pro-remain than anticipated? I would be very very worried if I were a British citizen in the remain camp right now.
Maybe be glad you don't have a betting account. I wouldn't trust anyone who thinks they can model this to the point of literally attaching "p=1.0", this early on.
I am not much of a gambler, but with a spread like this it looks very tempting. The currency market boys seem to think out has won.
Anyway using my imaginary betting account I will bet a million pounds - I will enjoy the cup of imaginary coffee I will be able to buy with my winnings :)
No I live in Australia so I have to covert my imaginary winnings pounds into imaginary Australian dollars to be able to buy my imaginary coffee ;)
More seriously the model was right and the bookies wrong - it is relatively rare that you get an opportunity like this handed to you to make money with almost no risk.
I still think p=1.0 was not possibly achievable via any model at that point in the counting, which means betting based on it would have been an enormous risk. The fact that the bet would have paid off makes no difference; especially given how close the vote seems to have gone, a coin flip would have gotten it right half the time, and I doubt you'd say that a coin flipper who just declares 100% confidence has a good model.
The model was almost exactly spot on, while the bookies were still offering 6/4 on for remain at the time I originally posted. The betting market really was not adjusting to the changing data and was way off on the real odds - this is quite rare.
I certainly hope that the people who made money off Chris’ model follow his suggestion and donate to the Jo Cox fund.
[0] http://www.ueapolitics.org/2016/06/23/eu-referendum-allnight...
[1] http://www.ueapolitics.org/2016/06/09/the-eu-referendum-what...
Edit: They moved their blog to Medium because of the demand: https://medium.com/@chrishanretty/eu-referendum-rolling-fore...