Why does the drop from p=0.32 to p=0.03 seem crazy/bogus? Isn't that what you'd expect as the probability distribution both narrows (with more evidence) and moves toward the 'leave' side?

It's a big jump in a single update. Given that Remain has since pulled ahead in the raw total, this only makes it appear more questionable.

That Remain pulled ahead is somewhat irrelevant though. That seems to be mostly due to London reporting in, which was always expected to vote Remain. What's relevant (for this model at least) is whether the results are higher or lower than predicted per area.

And now Leave is ahead again

If the model is being updated based on results as they come in, and the results coming in are not randomly distributed, then the updates will be of questionable value. In particular, this update came when a large number of predicted pro-leave results had come in, and no results from predicted strong pro-remain results had come in, so I'm not sure it has much value as a prediction.

Interesting he's since increased the certainty of a Leave vote even after a couple of unexpectedly strong pro-Remain votes swung the betting markets back in favour of Remain

Whether that's because he's better than the markets at modelling differential turnout or the markets know things his confirmed results data doesn't about predicted results in places like Birmingham remains to be seen...

As I understand it, the model is based on the difference between expected and actual results in each area. So the order that results come in should not affect the prediction.

> So the order that results come in should not affect the prediction.

This model uses a frequentist prediction interval, which assumes independently drawn samples, meaning reporting order must be random for the assumptions to be valid. If reporting is non-random, e.g. how early or late a district reports is correlated with things like region, demographics, population density, etc., then the prediction interval is probably narrower than it should be, especially early on in the reporting (meaning the model is overconfident in its prediction).

The headline prediction is more robust if you just want to know which outcome is more likely given current results, but the probabilities being badly calibrated due to these kinds of model assumptions is a common issue in quantitative polisci models.

That was P = 0.03, or 3%. Not 0.03%.

Yup! He did qualify his prediction though:

https://medium.com/@chrishanretty/eu-referendum-rolling-fore...

>This is a big update, and I'm conscious that I may have made a terrible mistake somewhere in estimating differential turnout, but here goes:

That update might be a bit dated... Glasgow just reported, heavily in favor of Bremain, tipping the scale of the current vote count.

The totals will go back and forth, but it's all about turnout proportionally in in- vs out- regions versus original projections. Glasgow was expected to be massively pro-remain but did Glasgow turn out in higher/lower numbers than anticipated? and did Glasgow go more or less pro-remain than anticipated? I would be very very worried if I were a British citizen in the remain camp right now.

Depends on how Glasgow was predicted to vote in their model. If it matches their prediction then the impact would be limited.