“How we got the final #Brexit poll wrong” – Ben Page of Ipsos Mori at #ijf17

I’m at the International Journalism Festival in Perugia, and here are my notes from a session called “Pollsters or astrologists? The social role of opinion polls in politics” which featured Ben Page, the CEO Ipsos Mori

“If I tell you there is a 20% chance of something bad happening when you leave the room, don’t leave the room”

Having covered a lot of opinion poll data during my time at Ampp3d, I was very much looking forward to hearing Ben Page, the CEO of Ipsos Mori talking about opinion polling during Brexit. He explained how the pollsters got it wrong.

He stared with a defence of the art, saying that if you look at all the opinion polls across all the elections that go on around the world, polling is still much more often right than wrong. His colleagues had recently got the Netherland’s election absolutely spot on.

He went on to say that on the morning of 23 June 2016, their final poll on the Brexit vote placed it at 51% for Leave. And they didn’t believe it. He explained the factors that made them mistrust the figure.

First of all, in referendums it is generally observed that as the vote nears, there is a “flight to safety” as people veer in support to the status quo. In the last two weeks of the Scottish Independence campaign, for example, he said they had seen a 55% Yes vote swing down to 45% in just ten days. It was logical to assume a similar thing would happen with Brexit.

Polling in Britain stops two days before an election. So they also have another couple of uncertainties to factor in. Not everybody they speak to on the Tuesday will actually vote on the Thursday. Whether through being hit by a bus, changing their mind, or it just being a bit rainy, they can guarantee that some of the people who said they will vote this way, won’t.

There is also a consistent figure of about 10% of British voters who only make up their mind which way they will vote in the polling booth. So they have a guesstimate how this 10% will act.

Ben reminded us that pollsters are human beings too, so they look for patterns. They had adjusted their model for the 2015 general election, because they been historically over-estimating the likelihood of Labour voters turning out at the polls. That adjustment worked for 2015, but applied to the 2016 referendum, got the answer wrong. Pollsters constantly adjust their methodology and the way they produce the final numbers, not because they are trying to mislead people, but because they are trying to be right.

Ben said that the public, politicians and journalists need to be much better at understanding probability and uncertainty. He explained that he went on the radio the morning of the vote and said “there’s a 40% chance that Brexit might happen”, but the media aren’t interested in that, they want a definitive answer. If Ben essentially says it is too close to call, the question that comes straight back at him is “Don’t you trust your own polling? Tell us the number!”

Still, there was one number that really stuck out at Ben’s presentation. He said that when the British public were polled about what they cared about when David Cameron came to power seven years ago, just 1 in 100 people thought the EU was the most important issue facing the country. And yet here we are, facing the prospect of talking about nothing but Brexit for years to come. Thanks, Dave. Well done mate.

You can watch Ben’s panel appearance here. Find all my blog posts from the 2017 International Journalism Festival.