The unexpected Conservative majority in the 2015 UK general election has led to considerable agonising in the polling industry. Why were the polls — which consistently predicted a hung parliament, or even a Labour victory — so wrong? A polling industry enquiry has come to some interim conclusions, here paraphrased by the BBC political editor, Laura Kuennsberg:
Pollsters didn’t ask enough of the right people how they planned to vote. Proportionately they asked too many likely Labour voters, and not enough likely Conservatives.
Nobody is suggesting that this bias was intentional — it was the accidental by-product of the polling methodology which, rather than drawing a random sample and then knocking on doors (the gold standard, but very expensive), relied heavily on telephone and online surveys. The problem with phone surveys, according to Martin Boon from ICM, is that out of 30,000 random numbers only around generally 2,000 agree to participate. This sample is anything but representative:
Adam Drummond, from Opinium, suggested that the people who agreed to be interviewed were so politically enthusiastic that the would even vote in elections for the European Parliament. . . This seems like a Groucho Marx problem — people answering the phone and agreeing to answer questions about their political opinions automatically means that they are too politically engaged to be representative.
Online polling is even worse:
Online polls are answered by a database of volunteers who have signed up to be on a panel and who the company knows a lot about. When the company is commissioned to do a poll it can be sure that the people it is asking have the same features as the whole population in — for example, the proportion of men and women, or the age profile or income distribution.
However what stratified sampling does not do is to control for the level of political engagement, so
The online polls came out with much the same inaccuracies as the phone ones. Does people’s willingness to be on a panel automatically make them unrepresentative?
Filed under: Elections, Opinion polling | 62 Comments »